Dense optic flow with discontinuity detection

Not a new algorithm; good ideas from several existing algorithms are combined.

Score on the Barron/Fleet/Beauchemin survey Yosemite benchmark: 
   full frame dense flow:  average error 6.04, std.dev 7.71.
   land region dense flow: average error 2.65, std.dev=1.87 (<1% error)
This is competitive with reported academic results (Details).
I do not know how it compares to the results of commercial packages such as ReTimer.


Sample movie



Quicktime for mac/win. (Sorenson3, may require Quicktime version5, download player from here )
Mpeg for win, unix. quality of the quicktime is better.

Blue overlay is painted on first frame and carried by the flow over subsequent frames.
Comment: this shows computing flow over a medium range of motions without drift. Many flow algorithms can only compute small motions, and the strategy of registering each frame to the next and composing the computed flow would result in noticable drift.

Examples

face frame 1
original frame 1 		frame 1 warped to frame 2
frame 1 warped to match 2 		face frame 2
original frame 2

face frame 1
original frame 1 		slomo frame 1.5
"slomo" frame 1.5 		face frame 2
original frame 2


Fake motion blur.




Benchmark Details:

The Barron/Fleet/Beauchemin error is the angle in degrees between ground truth and computed vectors; each vector is (u,v,1) normalized. This gives a measure of both directional and distance error in a single number.

The best reported dense flow score in the Barrons et.al. survey (mid 90s) was avgerr=11.26+std.dev=16.41 for a modified Horn-Schunck. A more competitive result is that from M.Black's code: avgerr=8.6,std.dev=9.3. The program described here scores avgerr=6.04+std.dev=7.71.

The Yosemite benchmark is CG rendered digital elevation data for which the correct ground truth flow is known. The sky, however, is a layered evolving Perlin noise thing that does not have a single correct flow velocity. Because of this errors on this benchmark can never be zero. Cropping out the sky region results in a score of avgerr=2.65+std.dev=1.87 for the current program.


One frame from Yosemite benchmark


Flow computed with discontinuity detection, average error 6.04.


Flow computed using M.Black algorithm for comparison, average error 8.6.