CS4243 – Computer Vision

During my stay at NUS I followed a course on Computer Vision by Terence Sim. The course was partly a project that I did with Florian Kock and Samuel Martin. Our self-assigned subject was an image stabilization algorithm.
This page shows you the result of our algorithm and the slides of the final presentation. The slides can be seen by clicking on the image to the right.

Below is a result video of our algorithm. The algorithm is based on removing the high-frequency movements found in video. In order to do so, three steps had to be taken.

Step 1: Finding the movement
In order to find the movement, we calculated the difference in position, rotation and zooming of every two succeeding frames. This was done by finding corresponding points in every two frames, selecting the best ones and calculating the homography.
Finding corresponding points was done with the Shi-Tomasi algorithm to find valuable features to track and the pyramidical Lucas-Kanade optical flow calculation.
Because the result was still pretty noisy, we iterated the calculation of the homography, removing the points that had a too large error at every step.

Step 2: Averaging the movement
For every frame a moving average was taken in order to remove the high-frequency movement. Because of the use of homographies, this was not as straightforward.
This lead to homographies that, once applied to the original frames, correct the movement by removing the unwanted movement, but keeping the intended movement.

Step 3: Compensating for the movement
In the last and final step, all the frames were transformed according to the calculated homographies. An extra step is taken by cropping the image so that the image is filled entirely in every frame.

The result
The result is shown below. The first frame is the uncorrected movement. In the middle frame the frames are corrected, it shows that every frame can be rotated, shifted and enlarged in order to compensate for the shaking. The last frame is the final result where a number of pixels are taken from the sides and top and bottom.