BACKCHANGE
Static background processor
A VirtualDub filter
by Krzysztof Wojdon, 2004
Table of Contents

1. WTF? (Why this filter?)
2. Download.
3. Configuration dialog box.
4. Short tutorial.
5. Script format definition.
6. Good things to come (but not yet, not yet).
7. Changes.
8. Author and usual yadayada.


1. WTF? (why this filter?)

During my last project I had to change a lot of backgrounds in animations. However, I had no access to original projects with layered graphics, so changing backgrounds meant precise manual painting in background areas of the video (because objects moving in front of background cannot be affected). For example: I had a group of animated characters with waving baloons in the wind and all of it in front of "Happy Birthday" poster that I had to change. Painting a few dozens of frames with a new sentence in place of "Happy Birthday" (with care for all ballons, baloon's strings and moving characters) could make interesting entry in my CV, but definitely would seriously impact my brain cells.

Nevertheless, for the first (simplest) animation I used classic "convert to frames and repaint each frame" method. One hour later it became clear that I was not going to finish all the animations before deadline. Things get worse when a few minutes later I realised that I used a slightly too dark green and animation did not match rest of the game. So I had to... repaint all frames. It was only 40 or 50 frames of simple animation, but that was enough. By that time I was sure that I needed some kind of process automatisation. And I wrote this filter.

One picture stands for 1024 words, as programmers say, so here we go. Imagine changing both texts in animation like this:


Easy stuff? Sure, there is nothing
better than 200 easy frames.

But believe me, getting result like this:


...is not a hard job


*If you want to see above animation in full size, you have both of them in tutorial folder.

PESIMIST'S NOTE: Unfortunately this program does not make miracles like eg. automatical reconstruction of the background (I wish it did!). You have to do it manually from the original frames, using uncovered parts of the background in different frames (just like you would with classical frame-by-frame painting). Good side is that with this filter you paint once, not for each and every frame.

Please note, that though name of the filter implies that you can change only background, it is also true that you can change any part of the image. Program does not differentiate between planes in any mystical way, so any part of video that is identical to the compare pattern (ie. old background) will be replaced. So when you put foreground in the background pattern image, you will change also this part of the video.



2. Download

As long as possible all my filters (and this one) will be kept at http://www.vander.prv.pl and this address will provide you with current builds. This .prv.pl domain is also quite useful, because it links dynamically to my current site, that can occasionaly move from one host to another. So if you want to link to my site, please use http://www.vander.prv.pl and not host address (like republika, onet, etc.).



3. Configuration dialog box

Script file name Click here, when you want to select script file. Compile selected script. You can test script syntax without closing configuration window. When this is marked, filter paints matching area with key color (in normal mode pixels from new background are used). This is usual boring stuff (short nonsense about program).
Keep mouse pointer for a moment over interesting part of the user interface to get short information.


All filter parameteres are changed from the script file. There is only one option that can be changed from filter configuration window:

Key color - this mode is useful if you want to check filter operation area. If this box is checked, filter does not replace affected pixels by new background pixels, but instead paint them with solid color. By affected I mean those that filter is going to change, not all pixels that filter works on. Key color used in the given frame is defined in script or (when left undefined) by default is bright purple (RGB=FF00FF).



4. Short tutorial

Let's assume that you have filter installed (if not - install this filter by copying file StatBgProc.vdf into plugins folder of VirtualDub and restart VirtualDub. You should also have following files in the tutorial folder:

  • anim_source.avi - source video clip
  • frame1.bmp & frame2.bmp - two frames taken from source video.
  • tut_script.txt - filter script for tutorial animation
  • backgrounds\background_new1.tga & backgrounds\background_new2.tga - two new replacement backgrounds
  • backgrounds\background_reconstructed.tga - reconstructed from frame1.bmp & frame2.bmp original background. I give it if you are not able (or do not want) to paint your own.

In this short tutorial I also assume that you are familiar with VirtualDub and general methods of image retouch and editing. If you do not know how to use clone tools or gradients to repaint images, you probably do not have to use this filter.

  1. Reconstructing background. This is hardest part of the whole process. If you only want to test filter in action, you can skip this point, all required result files are already present in the tutorial folder.

    Take any frames you need from the video, and using parts make one full background image without crosses. You can also use already selected images: frame1.bmp & frame2.bmp. Copying between these two frames is enough to get cross-free image (you can see my result in backgrounds\background_reconstructed.tga). Save result as 24-bit non compressed TGA file - backgrounds\background_reconstructed.tga. A little red color painting at the top and a simple clone tool action in the middle of the poster can quickly give you a clean board image, ready to handle new texts. Now you can modify reconstructed background in any way and save it as backgrounds\background_new1.tga. Change it again and save as a backgrounds\background_new2.tga. These modifications can be skipped, these two images are already prepared.

  2. Load tutorial anim_source.avi into VirtualDub. Add "Static Background Removal" filter in VirtualDub filter chain (menu Video/Filters, click on button "Add", select "static button processor" and click on "OK"). Now you see filter config dialog. Browse for script file (click on button "..."), find and select tutorial tut_script.txt file from the standard system file selection dialog. File tut_script.txt is placed in the same folder as anim_source.avi. If you fancy a little testing, you can click on "Parse script" button. Close configuration window with "OK" button. Close filter list with "OK".

  3. CONGRATULATIONS! You can move around video file in VirtualDub and see that output video has now new backgrounds with preserved foreground animation (crosses).


5. Script format definition

This filter is configured by the scripts. Script file is a text file in which user defines all the filter parameters. It is most versatile system and is most convenient (at least for me ;) ) when one is working with many parameters changing over time. But do not worry - script format is very simple. There is an example script in the tutorial folder - tut_script.txt. Please take a look at it, probably you will not have to read this paragraph when you see example script.

HINT: When you need a new script, do not write it from scratch. Use some other script (or tut_script.txt) as a template.

All empty lines and lines beggining with ';' character (comment lines) are skipped. Script is built from sections, section is called RANGE, because defines parameters for given frame range. Each range consist few lines of "KEYWORD=VALUE" syntax. Range definition ends with a "---" line. Each range describes (works on) frame range (from frame A to frame B) and filter parameters for all frames that are in this range. Below comes example of ONE range definition. This definition consists all possible keywords. There are only three keywords that are required in range definition: Frames, BGFile and BGFileNew are required, all other are optional.

Example of range definition:

Frames = 0-6
BGFile = backgrounds\background_reconstructed.tga
BGFileNew = backgrounds\background_new1.tga
KeyColor = FF00FF
Rect = 130,100,240,180
Threshold = 15
---


Frames - defines frame range (start-end) for which following parameters are valid. This parameter must be declared in range definition.

BGFile - defines filename of original background file. Frame pixels identical to pixels from this image will be replaced. This file must be saved as uncompressed 24-bit Targa (TGA) file. This parameter must be declared in range definition. File path is relative to the script path. Eg. when script file is "c:\anim1\demo\script.txt", then image "bg\new1.tga" points to file "c:\anim1\demo\bg\new1.tga".

BGFileNew - filename of new (replacement) background. Path is relative to the script path. Pixels from this image will be put in place of old background. This file must be saved as uncompressed 24-bit Targa (TGA) file. This parameter must be declared in range definition.

KeyColor - RGB color definition of the color which will be painted, when key coloring mode is enabled in filter configuration. This parameter is optional; default key color is RGB = FF00FF.

Rect - Defines coordinates of the processing rectangle. Can be useful for speeding up operation, when only specified small parts of the video will be changed or when somebody does not want to reconstruct whole background. Coordinates are defined as: first pair is X,Y of top-left corner, and second pair is X,Y of bottom-right rectangle corner. This parameter is optional; default range is full frame. In tutorial animation, last part of animation is rendered with defined rectangle, so there is only small part of the background changed (see movie titles in the middle of the frame).

Threshold - Maximum difference below all pixel RGB comparisons are considered "these pixels are the same". This value is used during comparison between RGB of background and video pixels. We need this one, because in video processing signal is almost never stored with 100% precision (beside really small number of REALLY loseless compressors). So comparison would miss many pixels when comparing in strict arithmetical meaning. Setting this value too low will result in skipping pixels of the background that should be changed; going too high will start to paint over foreground layer. This parameter is optional; default threshold is 0.



6. Things to do

Currently there is one feature I would like to add.
  • Script change detection based on script file date/time modification time. This way any script change would automatically call filter's script parser on the next frame render.
Further developement of the filter depends on the user feed back. If you think there are features, that could make this tool more useful, please contact me at krzysiekwojdon@gmail.com. Please use some meaningful mail subject (like something with filter name, VirtualDub, etc.) so I will not miss your e-mail in the jungle of spam.


7. Changelog

Build 1.0

This is first initial release. This filter was successfully used in real life project and whole filter idea worked well for me. The implementation may be slow (simple algorithm, no optimisations). This build may still have bugs, because some parts were added and/or fixed after project was finished (and as a result - those features are not tested).


8. Author and yada yada

This program was entirely written by Krzysztof Wojdon (that's me, yoohoo!) and all copyrights remain in ownership of the author (yes, that's also me). This program is freeware and can be used for all (non-commercial and commercial) purposes. The filter is given AS IS. Author does not guarantee that this program fits users purposes or works as expected (eg. sometimes it could look like space flight simulator, you never know). Also I do not take resposibility for your computer, hard drive, girlfriend, dog and cat if something goes wrong. However, I cannot see how this program could do anything like this. Side note: I also think this kind of disclaimers suck and mean anything only to lawyers (and I'd prefer not meet any in this life). Use common sense and make this world a better place.

As usual, BIG CREDITS go to Avery Lee for his excellent utility: VirtualDub.