Dense optic flow with discontinuity detection

Not a new algorithm; good ideas from several existing algorithms are combined.

Score on the Barron/Fleet/Beauchemin survey Yosemite benchmark:
   full frame dense flow:  average error 6.04, 7.71.
   land region dense flow: average error 2.65, (<1% error)
This is competitive with reported academic results (Details).
I do not know how it compares to the results of commercial packages such as ReTimer.

Sample movie

Quicktime for mac/win.
(Sorenson3, may require Quicktime version5, download player from here )
Mpeg for win, unix. quality of the quicktime is better.

Blue overlay is painted on first frame and carried by the flow over subsequent frames.
Comment: this shows computing flow over a medium range of motions without drift. Many flow algorithms can only compute small motions, and the strategy of registering each frame to the next and composing the computed flow would result in noticable drift.


face frame 1
original frame 1
frame 1 warped to frame 2
frame 1 warped to match 2
face frame 2
original frame 2

face frame 1
original frame 1
slomo frame 1.5
"slomo" frame 1.5
face frame 2
original frame 2

Fake motion blur.

Benchmark Details:

The Barron/Fleet/Beauchemin error is the angle in degrees between ground truth and computed vectors; each vector is (u,v,1) normalized. This gives a measure of both directional and distance error in a single number.

The best reported dense flow score in the Barrons survey (mid 90s) was for a modified Horn-Schunck. A more competitive result is that from M.Black's code: avgerr=8.6, The program described here scores

The Yosemite benchmark is CG rendered digital elevation data for which the correct ground truth flow is known. The sky, however, is a layered evolving Perlin noise thing that does not have a single correct flow velocity. Because of this errors on this benchmark can never be zero. Cropping out the sky region results in a score of for the current program.

One frame from Yosemite benchmark

Flow computed with discontinuity detection, average error 6.04.

Flow computed using M.Black algorithm for comparison, average error 8.6.