Camera Tracking can be the bane of any VFX artist.
All of us can relate to the anxiety felt when you open up a shot that needs tracking only to be faced with a smorgasbord of less-than-optimum conditions: blurry footage, excessive motion blur, no real details, everything that makes our tracking job exceedingly difficult.
When tasked with any camera tracking, I always follow the same steps that allow me to come away with a solid track time and time again. I’m going to outline these so that you you too can be set up for tracking success.
I’ll be demonstrating these techniques in Blender and After Effects, but the principles should be fairly consistent across the board no matter what software you use.
Blender Camera Tracking
The Blender camera tracker has been around for a while, and whilst it can be finicky at times and isn’t always perfect, if you are putting together your VFX scene in Blender, you should try and pull a solid track using the internal camera tracker.
Firstly, you’ll want to import your footage into the movie clip editor and cache as much of it as possible into Blender by clicking 'Prefetch'.
This is a fairly scattergun approach, however, I’ve found that it generally avoids spending hours and hours manually selecting and refining individual tracking points.
With the playhead set at the first frame, select ‘Detect Features’ from the toolbar on the left-hand side. This will automatically detect numerous potential tracking points in the frame.
You can tweak the margin, threshold and distance to make even more markers appear. The more the merrier at this stage, so crank it up.
Hit the track forward button.
Blender will aim to track these points as the clip plays through frame by frame. At this stage don’t worry if a number fail or slip out of place, this is to be expected as we’ve cast such a wide net.
Once a significant number of markers have failed, pause the track. Hit ‘Detect features’ again, and rinse and repeat.
Keep doing this until there are a healthy number of tracked markers throughout the clip. You can even track backwards through frames if needed.
At this stage everything shall appear to be a bit of a mess.
Let’s try and solve the camera. But before we do, if you know the lens and sensor size of your camera that you shot the footage with, add it in the panel on the right. In addition to this, right above the Solve Camera Motion button, check the ‘Keyframe’ box. This will let Blender automatically select the keyframes with which to track the scene.
Click ‘Solve Camera Motion’.
Blender will do its thing; it may take a while depending on the length of the clip. Once it’s done, you will be able to see the average solve error. The end goal for this value is to be roughly under 0.5 pixels, anything under that is a seriously good camera track! However right now you’re probably staring at a value of 100 pixels or above. Yikes.
But that is okay. Let’s clean it up.
To get an idea just how much chaos you’ve created, open up a new panel with the movie clip editor and set it to show the graphs of all your tracks.
You’ll likely see a bunch of the lines of your graph erratically moving away from the overall general pattern, these are just some of the outliers we need to eradicate.
Back in your tracking panel, head over to the ‘Clean Up’ drop-down and click ‘Filter Tracks’. If you set this to something high between 10-15, Blender will automatically select all the most erroneous tracking points. Hit Delete.
Now solve the camera motion again.
Hopefully this has improved your average solve error, but we will likely need to improve it some more.
Now under, ‘Clean Up’, click ‘Clean Tracks’ and adjust the reprojection error to something like 2 or 3, this will select all the tracks that shift more than 2 or 3 pixels away from what would be expected.
Also don’t be afraid to eyeball the markers and delete any that are clearly doing something unexpected. For example, they may be tracking something moving, or they may be sliding around as something in the foreground obscures the original point.
Try another camera solve; you can also refine the focal length, optical centre and radial distortion to help dial it in.
When the solve is nicely sitting around half a pixel, you can click 'Set Up Tracking Scene' under Scene Setup, and Blender will nicely set up everything you need in the 3D view ready to add your visual effects elements.
If at the end of this process, you still don’t have a solid camera track, there are a couple of other quick and dirty tricks that can help!
For this, we need to dive into Adobe After Effects…
After Effects + Blender Camera Tracking - NO PLUGINS or ADDONS
The good folks at Adobe have actually blessed us with arguably the easiest and fastest camera tracking solution. The internal camera tracker can make light work of most video clips.
This may not be helpful if we need to build up our scene in Blender though, so let’s look at a way that we can use this After Effects tool to benefit our Blender track.
With your video clip in a composition, move over to the tracker panel and select ‘Track Camera’, make sure you select the ‘Detailed Analysis’ checkbox before it gets going. After Effects will begin its tracking work.
Eventually the clip’s track will be solved and you’ll see a whole bunch of tracking markers in the scene.
You are able to right click on any of these markers (or range of markers) and choose ‘Create Solid and Camera’. This will create a solid square right in the scene that should, hopefully, be perfectly tracked in.
How this can benefit you, is by giving you far more trackable points with which to use in the Blender camera tracker.
Create as many of these solids as possible in different vibrant colours, then export your clip out of After Effects (preferably in ProRes) to bring into Blender.
Now, you can manually choose the corners of these solids and track those, or revert to the automated method above!
These techniques serve as a guide for how you can accomplish camera tracking easier in Blender, with a little helping hand from After Effects.
The methods do not utilise any third-party plugins or addons; if you’d like to see a helpful tutorial post about some plugins (free and paid), that can also help you out, let us know by tagging us on our socials!
