search

JuliaImages / ImageDistances.jl

Distances between N-dimensional images
https://github.com/JuliaImages/Images.jl
Other
15 stars 8 forks source link

Performance Benchmark #66

Closed ashwani-rathee closed 3 years ago

ashwani-rathee commented 3 years ago

Notes:

johnnychen94 commented 3 years ago

print_table.jl in Distances.jl doesn't seems to work on my system

Maybe we can instead use PkgBenchmark?

ashwani-rathee commented 3 years ago

Yeah,that would be great choice since it basically does exactly what we did here in a much more cleaner way:

julia> benchmarkpkg(ImageDistances)
PkgBenchmark: Running benchmarks...
┌ Warning: Usage of "lena" is not recommended, and the image may be removed in a later release. See
│ https://womenlovetech.com/losing-lena-why-we-need-to-remove-one-image-and-end-techs-original-sin/ for more information.
└ @ TestImages /home/ashwani/.julia/packages/TestImages/mpUUF/src/TestImages.jl:89
PkgBenchmark: creating benchmark tuning file /home/ashwani/julia-related/ImageDistances.jl/benchmark/tune.json...
(1/2) tuning "colwise"...
  (1/3) tuning SqEuclidean...
    (1/1) tuning "specialized"...
    done (took 5.898575339 seconds)
  done (took 6.570366653 seconds)
  (2/3) tuning Euclidean...
    (1/1) tuning "specialized"...
    done (took 5.672162934 seconds)
  done (took 6.154629801 seconds)
  (3/3) tuning Cityblock...
    (1/1) tuning "specialized"...
    done (took 5.708270225 seconds)
  done (took 6.215746567 seconds)
done (took 19.521080364 seconds)
(2/2) tuning "pairwise"...
  (1/3) tuning SqEuclidean...
    (1/1) tuning "specialized"...
    done (took 5.707549161 seconds)
  done (took 6.161534441 seconds)
  (2/3) tuning Euclidean...
    (1/1) tuning "specialized"...
    done (took 5.723401211 seconds)
  done (took 6.174857527 seconds)
  (3/3) tuning Cityblock...
    (1/1) tuning "specialized"...
    done (took 5.680195716 seconds)
  done (took 6.124653153 seconds)
done (took 18.911596809 seconds)
Tuning 100%|███████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| Time: 0:00:40
(1/2) benchmarking "colwise"...
  (1/3) benchmarking SqEuclidean...
    (1/1) benchmarking "specialized"...
    done (took 2.324882216 seconds)
  done (took 2.78765212 seconds)
  (2/3) benchmarking Euclidean...
    (1/1) benchmarking "specialized"...
    done (took 2.29229207 seconds)
  done (took 2.750039344 seconds)
  (3/3) benchmarking Cityblock...
    (1/1) benchmarking "specialized"...
    done (took 2.446123569 seconds)
  done (took 2.907767009 seconds)
done (took 9.03810426 seconds)
(2/2) benchmarking "pairwise"...
  (1/3) benchmarking SqEuclidean...
    (1/1) benchmarking "specialized"...
    done (took 5.512861716 seconds)
  done (took 5.968031401 seconds)
  (2/3) benchmarking Euclidean...
    (1/1) benchmarking "specialized"...
    done (took 5.654994267 seconds)
  done (took 6.244157039 seconds)
  (3/3) benchmarking Cityblock...
    (1/1) benchmarking "specialized"...
    done (took 5.580113373 seconds)
  done (took 6.091047563 seconds)
done (took 18.753209574 seconds)
Benchmarking 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| Time: 0:00:28
Benchmarkresults:
    Package: /home/ashwani/julia-related/ImageDistances.jl
    Date: 15 May 2021 - 21:19
    Package commit: dirty
    Julia commit: 6aaede
    BenchmarkGroup:
        2-element BenchmarkTools.BenchmarkGroup:
          tags: []
          "colwise" => 3-element BenchmarkTools.BenchmarkGroup:
                  tags: []
                  "Cityblock" => 1-element BenchmarkTools.BenchmarkGroup:
                          tags: []
                          "specialized" => Trial(182.747 μs)
                  "SqEuclidean" => 1-element BenchmarkTools.BenchmarkGroup:
                          tags: []
                          "specialized" => Trial(168.426 μs)
                  "Euclidean" => 1-element BenchmarkTools.BenchmarkGroup:
                          tags: []
                          "specialized" => Trial(164.271 μs)
          "pairwise" => 3-element BenchmarkTools.BenchmarkGroup:
                  tags: []
                  "Cityblock" => 1-element BenchmarkTools.BenchmarkGroup:
                          tags: []
                          "specialized" => Trial(730.735 μs)
                  "SqEuclidean" => 1-element BenchmarkTools.BenchmarkGroup:
                          tags: []
                          "specialized" => Trial(633.695 μs)
                  "Euclidean" => 1-element BenchmarkTools.BenchmarkGroup:
                          tags: []
                          "specialized" => Trial(656.023 μs)
johnnychen94 commented 3 years ago

and export_markdown for a pretty table print.

johnnychen94 commented 3 years ago

Given that ImageDistances is a wrapper package of Distances, it also worth benchmarking how much overhead we have here. For example, the time spent when eltype(img) is Gray{Float32}/Gray{N0f8} and Float32/N0f8(calls Distances). If there's a large gap when we definitely need to improve it.

ashwani-rathee commented 3 years ago

Yeah, export_markdown looks pretty clean. https://gist.github.com/ashwani-rathee/209b51ff69e001829c35dfb66c1c2765 Okay, I'll try to figure out the overheads.

ashwani-rathee commented 3 years ago

Since, It uses https://github.com/JuliaImages/ImageContrastAdjustment.jl/pull/37/files only. @johnnychen94 can you add the label and test if it works. I am not too sure how it works yet but I have added relevant files I think.

johnnychen94 commented 3 years ago

:tada: The error is expected because we don't yet have benchmark/benchmarks.jl file in the master branch.

To save CI resource I'm removing the run benchmark label. When this is done, I'll test it locally and then merge it.

ashwani-rathee commented 3 years ago

https://gist.github.com/ashwani-rathee/f447ff66d03677feee6127bf778f8689 Does this look like what we are looking for? Minkowski, Hausdorff still needs to be added to the benchmark though

ashwani-rathee commented 3 years ago

@johnnychen94 Can you check if the format shown in gist above works for us or not?

johnnychen94 commented 3 years ago

Yes! It looks good! Is it ready for you?

Apparently, we have some performance gaps to Distances.jl 😢

ashwani-rathee commented 3 years ago

Yeah. Now, It looks somewhat ready to me. Please review

I guess we need to cover up the gaps :)

johnnychen94 commented 3 years ago

Let's see how it works on the master branch.