Error Installing on Linux Julia 1.0.1 (2018-09-29)

[abcsds]

Running add https://github.com/JuliaImages/ImageTracking.jl

  Updating registry at `~/.julia/registries/General`
  Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Updating git-repo `https://github.com/JuliaImages/ImageTracking.jl`
[ Info: Assigning UUID 45f89549-7fd0-50e3-8bca-46e418af1b79 to ImageTracking
 Resolving package versions...
ERROR: Unsatisfiable requirements detected for package AxisAlgorithms [13072b0f]:
 AxisAlgorithms [13072b0f] log:
 ├─possible versions are: [0.1.0-0.1.6, 0.2.0, 0.3.0] or uninstalled
 ├─restricted to versions 0.2.0-0.2 by ImageTracking [45f89549], leaving only versions 0.2.0
 │ └─ImageTracking [45f89549] log:
 │   ├─possible versions are: 0.0.0 or uninstalled
 │   └─ImageTracking [45f89549] is fixed to version 0.0.0
 └─restricted by julia compatibility requirements to versions: 0.3.0 or uninstalled — no versions left

[zygmuntszpak]

Thanks for taking the time to report this. The package currently doesn't work on Julia versions greater than 0.6.4. We paused the development of this package for a while because we were waiting for various packages that we depend on to upgrade to versions greater than 0.6.4.

Anyway, given that you are interested in using this package I reckon now is a good time to bring it up to date. Hence, I'll start working on upgrading this package---stay tuned.

[zygmuntszpak]

I've started working on this in earnest again today. I've managed to get rid of all of the deprecation warnings for the Lukas-Kanade implementation. However, it seems that the upgrade has broken our support for OffsetArrays somehow as one of the tests now errors. I hope that we will have the Lukas Kanade and Farneback optical flow running properly by the end of the week.

Which particular algorithm are you intending to use?

[zygmuntszpak]

Here is a link to the issue that is causing one of the tests to fail now: https://github.com/JuliaImages/ImageFiltering.jl/issues/82

[zygmuntszpak]

I've fixed JuliaImages/ImageFiltering.jl#82 and the tests for Lukas-Kanade now all pass on my machine. Moving on to Farneback...

[zygmuntszpak]

Apart from getting rid of deprecation warnings I've also implemented a fast (convolution) based version of the polynomial expansion step that is utilized in Farneback. Apart from that I've rewritten the core Farneback optical flow algorithm multiple times looking for ways (with and without StaticArrays) to reduce the number of allocations and improve the run-time performance.

At the moment what is killing performance is the several rounds of smoothing of the polynomial coefficients. Perhaps we can only expect this algorithm to perform well if the convolutions are all done on the GPU.

I've decided to clean up the API a bit as well and discovered that occasionally we get NaNs for the flow vector. Still need to determine what the cause is.

I intend to push an experimental 1.0 branch sometime this weekend so that you (whoever is reading these comments) can help test it out. Please let me know if you are interested.

It would be great to write a script to visualize the flow vectors and verify that we get sensible results.

[abcsds]

Thank you so much! I am interested and I'll be testing it with videos from the DEAP dataset. I also have access to a GTX1080Ti, in case GPU convolutions are a feature.

[zygmuntszpak]

The tests for Lucas-Kanade and Farneback now all pass on the 1.0 pull-request. I spent some time refactoring the API so that there will hopefully less chance of a breaking change in the near future. I also restructured the tests. At the moment I'm doing another pass over the Lucas-Kanade code to reduce the number of allocations and try to fix type instabilities.

[zygmuntszpak]

I should also mention that the code still needs a lot of refactoring. I have deliberately not requested a code review yet for that reason. My main aim at this point was to mangle the previous version into 1.0 compatibility and to try to bring performance up to a reasonable level. After this next pass over the Lucas-Kanade code I will merge an updated version of the 1.0 pull-request. That will allow you to at least use the code. Thereafter, I will focus on cleaning the implementation.

I'll also create a separate issue for a roadmap of things that need to get done before we can register the package (e.g. examples and documentation).

[abcsds]

Thank you so much for your work. It is greately apreciated. If I could install the code I could do tests and documentation. I see the REQUIRE includes AxisAlgorithms 0.2.0 0.3.0, but Pkg still declares:

restricted to versions 0.2.0-0.2 by ImageTracking [45f89549], leaving only versions 0.2.0

Could this be a Pkg error?

[zygmuntszpak]

I've merged the 1.0 compatibility branch with the master branch. If you check out the master branch and type ] test ImageTracking you should see all the tests pass.

I will write a quick demo with a better explanation later today, but here is a short snippet to see the usage.

using Images, TestImages, StaticArrays, ImageTracking, Random, OffsetArrays, CoordinateTransformations, BenchmarkTools
img1 = Gray{Float64}.(testimage("mandrill"))
trans = Translation(0, -3.0)
img2 = warp(img1, trans, axes(img1))

corners = imcorner(img1, method=shi_tomasi)
I = findall(!iszero, corners)
r, c = (getindex.(I, 1), getindex.(I, 2))
points = map((ri, ci) -> SVector{2}(Float64(ri), Float64(ci)), r, c)

flow, indicator  = optical_flow(img1, img2, points, LucasKanade( 11, 4, 20, 0.000001))

The indicator array flags whether a point in points was tracked or not.

So using the above example:

 points[1250]
2-element SArray{Tuple{2},Float64,1,2}:
 269.0
  44.0

indicator[1250]
true

flow[1250]
2-element SArray{Tuple{2},Float64,1,2}:
 3.111466892469217e-8
 3.000001117619417

If we set

r,c = Int.(points[1250])
p,q = Int.(points[1250] + round.(flow[1250]))

then the corresponding intensity of img1[r,c] is given by img2[p,q].

[abcsds]

Installed and tested!

[ Info: ("Case: Horizontal Motion", "Percentage Error: 3.1848907470703125", "Maximum Error: 1.737930059702624")
[ Info: ("Case: Horizontal Motion", "Percentage Error: 3.1848907470703125", "Maximum Error: 1.737930059702624")
[ Info: ("Case: Vertical Motion", "Percentage Error: 3.073883056640625", "Maximum Error: 2.2299089722243655")
[ Info: ("Case: Vertical Motion", "Percentage Error: 3.073883056640625", "Maximum Error: 2.2299089722243655")
[ Info: ("Case: Combined Motion", "Percentage Error: 3.737640380859375", "Maximum Error: 1.6529357942236553")
[ Info: ("Case: Combined Motion", "Percentage Error: 3.737640380859375", "Maximum Error: 1.6529357942236553")
Test Summary: | Pass  Total
Farneback     |   15     15
[ Info: ("Case: Horizontal Motion", "Percentage Error: 0.0", "Maximum Error: 0.02210893745418784")
[ Info: ("Case: Horizontal Motion", "Percentage Error: 0.0", "Maximum Error: 0.02210893745418784")
[ Info: ("Case: Vertical Motion", "Percentage Error: 0.0", "Maximum Error: 0.02707811875208793")
[ Info: ("Case: Vertical Motion", "Percentage Error: 0.0", "Maximum Error: 0.02707811875208793")
[ Info: ("Case: Combined Motion", "Percentage Error: 0.0", "Maximum Error: 0.02120687328850489")
[ Info: ("Case: Combined Motion", "Percentage Error: 0.0", "Maximum Error: 0.02120687328850489")
Test Summary: | Pass  Total
Lucas-Kanade  |    6      6
No features of given type found in region!
Test Summary: | Pass  Total
haar          |   19     19
   Testing ImageTracking tests passed

I'm testing the LK now with the DEAP dataset Should I close this issue?

[zygmuntszpak]

Yes, I guess we can close this issue. Now that you are testing I'm sure you will open a whole lot of new ones :D.

I looked at the website of the DEAP dataset. It doesn't contain any "ground-truth" optical flow does it? How are you going to determine that the flow is right? Could you describe your use-case a bit more before you close this issue?