JuliaImages / ImageFiltering.jl

Julia implementations of multidimensional array convolution and nonlinear stencil operations
Other
99 stars 51 forks source link

rework gabor filter and add log gabor filter #235

Open johnnychen94 opened 2 years ago

johnnychen94 commented 2 years ago

The previous Kernel.gabor function has three drawbacks:

benchmark on gabor filter:

using ImageFiltering

# new lazy Gabor array
kern = Kernel.Gabor((256, 256), 30, 0)
@btime collect($kern); # 1.157 ms (2 allocations: 1.00 MiB)

# old gabor function
σ, θ, λ, γ, ψ = 10, 0, 30, 0.5, 0
@btime Kernel.gabor(256,256,σ,θ,λ,γ,ψ); # 2.911 ms (51 allocations: 12.60 MiB)

Docs Preview: https://juliaimages.org/ImageFiltering.jl/previews/PR235/demos

TODO:

The implementation of the log gabor filter heavily follows https://www.peterkovesi.com/matlabfns/PhaseCongruency/Docs/convexpl.html so I'm wondering if @peterkovesi could help review this.

codecov[bot] commented 2 years ago

Codecov Report

Merging #235 (e0f182a) into master (424523c) will increase coverage by 0.30%. The diff coverage is 100.00%.

:exclamation: Current head e0f182a differs from pull request most recent head b1981ea. Consider uploading reports for the commit b1981ea to get more accurate results Impacted file tree graph

@@            Coverage Diff             @@
##           master     #235      +/-   ##
==========================================
+ Coverage   92.14%   92.45%   +0.30%     
==========================================
  Files          12       14       +2     
  Lines        1642     1709      +67     
==========================================
+ Hits         1513     1580      +67     
  Misses        129      129              
Impacted Files Coverage Δ
src/kernel.jl 99.02% <ø> (-0.21%) :arrow_down:
src/gaborkernels.jl 100.00% <100.00%> (ø)
src/kernel_deprecated.jl 100.00% <100.00%> (ø)

Continue to review full report at Codecov.

Legend - Click here to learn more Δ = absolute <relative> (impact), ø = not affected, ? = missing data Powered by Codecov. Last update 424523c...b1981ea. Read the comment docs.

johnnychen94 commented 2 years ago

I'm marking this as ready as I don't plan to add any further major changes to this PR. I'm going to add phase congruency filter next, and then finally the FSIM quality index https://github.com/JuliaImages/ImageQualityIndexes.jl/issues/25. I'll keep this PR pending until everything downstream is ready.

ImagePhaseCongruency.jl is really a good reference but it has a strong MATLAB legacy, I believe we can get things faster and give it better composability by properly resolving and designing the function.

johnnychen94 commented 2 years ago

Benchmark shows that F32 on LogGabor is somewhat slow. I'll try to investigate it before merging this.

    BenchmarkGroup:
        2-element BenchmarkTools.BenchmarkGroup:
          tags: []
          "Gabor" => 6-element BenchmarkTools.BenchmarkGroup:
                  tags: []
                  "Gabor_ComplexF32_19×19" => Trial(5.625 μs)
                  "Gabor_ComplexF32_256×256" => Trial(994.030 μs)
                  "Gabor_ComplexF32_512×512" => Trial(3.857 ms)
                  "Gabor_ComplexF64_19×19" => Trial(7.870 μs)
                  "Gabor_ComplexF64_256×256" => Trial(1.440 ms)
                  "Gabor_ComplexF64_512×512" => Trial(5.553 ms)
          "LogGabor" => 6-element BenchmarkTools.BenchmarkGroup:
                  tags: []
                  "LogGabor_ComplexF64_19×19" => Trial(13.903 μs)
                  "LogGabor_ComplexF64_256×256" => Trial(2.422 ms)
                  "LogGabor_ComplexF64_512×512" => Trial(9.884 ms)
                  "LogGabor_ComplexF32_19×19" => Trial(68.269 μs)
                  "LogGabor_ComplexF32_256×256" => Trial(11.896 ms)
                  "LogGabor_ComplexF32_512×512" => Trial(51.113 ms)

Writing benchmark isn't something pleasant, but when you get the report, oh that's a surprise!

peterkovesi commented 2 years ago

I am very pleased and impressed that this work is being done. Please help yourself to any code you think is useful in ImagePhaseCongruency I am sure much of it can be considerably improved and made more interoperable with the rest of the Images ecosystem.

Another function from ImagePhaseCongruency that I think is deserving of being included in Images is perfft2(). It is very useful when one wants to perform filtering in the frequency domain with minimal artifacts introduced from the boundary discontinuities. See https://peterkovesi.github.io/ImagePhaseCongruency.jl/dev/examples/#Fourier-transform-of-Moisan's-periodic-image-component-1

Nate that I am now retired and find myself not doing very much coding these days. Indeed at the moment I am often off the grid for extended periods of time. Thus I am not in a position to provide code reviews but should you have any questions about my code I will be more than happy to try to answer them. In the meantime I will be cheering you on from the sidelines!

johnnychen94 commented 2 years ago

@peterkovesi Thanks for the information! I've made an initial implementation of the phasecong3 and it turns out that I only get 2x slower 😂 because of the repeated computation as I noted in https://github.com/JuliaImages/ImageFiltering.jl/pull/235#discussion_r741936206. The idea of separating the log Gabor frequency filter and the log Gaussian angular filter here to decouple the orientation and scale loops is really really insightful.

Profiling my initial implementation code shows that almost 80% of the computation is spent on the log Gabor filter, which is definitely a performance hotspot.

That said, this PR is far from ready or satisfying.