JuliaImages / ImageTracking.jl

Other
26 stars 8 forks source link

Adds farneback optical flow algorithm #3

Closed arijitkar98 closed 6 years ago

arijitkar98 commented 6 years ago

This is the basic implementation of the algo. I will keep working on the correlation and pyramidal part.

codecov[bot] commented 6 years ago

Codecov Report

Merging #3 into master will increase coverage by 1.63%. The diff coverage is 95.58%.

Impacted file tree graph

@@            Coverage Diff             @@
##           master       #3      +/-   ##
==========================================
+ Coverage   90.25%   91.89%   +1.63%     
==========================================
  Files           2        3       +1     
  Lines         154      222      +68     
==========================================
+ Hits          139      204      +65     
- Misses         15       18       +3
Impacted Files Coverage Δ
src/optical_flow.jl 100% <ø> (ø) :arrow_up:
src/farneback.jl 95.58% <95.58%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 875177d...94b469f. Read the comment docs.

zygmuntszpak commented 6 years ago

Benchmarking this on one of test-cases shows that the code is allocating way too much memory inside critical loops resulting in really slow code:

 54.493782 seconds (145.13 M allocations: 95.058 GiB, 25.05% gc time)

I'm experimenting with some changes to improve the performance.

zygmuntszpak commented 6 years ago

If we assume that the confidence parameter is constant then we can move several key matrix multiplications outside of the main loop. After making such a change and removing some other unnecessary allocations, I've been able to substantially improve the benchmark results.:

  5.416388 seconds (34.69 M allocations: 3.721 GiB, 15.70% gc time)

This is still relatively slow. Our future work should probably focus on implementing the pyramidal convolution-based method in order to obtain good run-time performance. Our current implementation can at least serve as a reference of 'correctness' for when we attempt the convolution-based implementation.

I'll post some of the requisite changes on our Slack channel so that you can incorporate them and modify this pull-request.

zygmuntszpak commented 6 years ago

I believe I understand how to implement the convolution-based polynomial approximation method. They key equations are 4.9 and 4.10 on the page 27 (page 35 in the PDF) of the thesis Multiscale curvature estimation polynomial expansion.

I will work on it after you finalize the current pull-request.