Some operations on `OpenCV.Mat` are slow and taking high memory.

Some array operations like findall slow and using high memory on OpenCV.Mat compared Array{T, 3}. Here is MWE

using Random
using BenchmarkTools
using OpenCV

Random.seed!(1234)

mat = rand(UInt8(0):UInt8(255), 3, 2^10, 2^10)
ocmat = OpenCV.Mat(mat)

f = >=(UInt8(100))

@assert findall(f, mat) == findall(f, ocmat)
@assert findall(f, mat) == findall(f, ocmat.data)
@assert findall(f, mat) == findall(f, ocmat.data_raw)

@benchmark findall($(f), $(mat))

@benchmark findall($(f), $(ocmat))

@benchmark findall($(f), $(ocmat.data))

@benchmark findall($(f), $(ocmat.data_raw))

Results

julia> @benchmark findall($(f), $(mat))
BenchmarkTools.Trial: 269 samples with 1 evaluation.
 Range (min … max):  15.501 ms … 44.234 ms  ┊ GC (min … max): 3.47% … 3.38%
 Time  (median):     17.868 ms              ┊ GC (median):    8.16%
 Time  (mean ± σ):   18.622 ms ±  3.149 ms  ┊ GC (mean ± σ):  8.28% ± 1.30%

         ██▆▃ ▁
  ▄▁▁▁▁▁▆██████▅▁▁▁▁▄▁▁▁▁▅▁▁▁▁▁▄▄▅▁▁▄▁▁▁▁▁▁▄▅▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▆ ▅
  15.5 ms      Histogram: log(frequency) by time      31.9 ms <

 Memory estimate: 44.25 MiB, allocs estimate: 6.

julia> @benchmark findall($(f), $(ocmat))
BenchmarkTools.Trial: 2 samples with 1 evaluation.
 Range (min … max):  2.491 s …   2.524 s  ┊ GC (min … max): 0.79% … 0.78%
 Time  (median):     2.507 s              ┊ GC (median):    0.78%
 Time  (mean ± σ):   2.507 s ± 23.730 ms  ┊ GC (mean ± σ):  0.78% ± 0.01%

  █                                                       █
  █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁
  2.49 s         Histogram: frequency by time        2.52 s <

 Memory estimate: 542.59 MiB, allocs estimate: 26367765.

julia> @benchmark findall($(f), $(ocmat.data))
BenchmarkTools.Trial: 252 samples with 1 evaluation.
 Range (min … max):  15.296 ms … 46.063 ms  ┊ GC (min … max): 2.31% … 6.62%
 Time  (median):     18.633 ms              ┊ GC (median):    8.80%
 Time  (mean ± σ):   19.874 ms ±  3.541 ms  ┊ GC (mean ± σ):  8.27% ± 1.33%

             ▆█▇▄▄▁
  ▄▄▁▁▁▁▁▁▁▁▇██████▆█▄▄▆▄▇▁▆▇▄▁▄▄▁▄▁▆▁▁▁▄▁▁▆▄▄▁▄▄▆▇▆▆▄▄▁▁▄▁▄▆ ▆
  15.3 ms      Histogram: log(frequency) by time        30 ms <

 Memory estimate: 44.25 MiB, allocs estimate: 6.

julia> @benchmark findall($(f), $(ocmat.data_raw))
BenchmarkTools.Trial: 239 samples with 1 evaluation.
 Range (min … max):  16.272 ms … 62.863 ms  ┊ GC (min … max): 9.46% … 2.66%
 Time  (median):     19.361 ms              ┊ GC (median):    8.68%
 Time  (mean ± σ):   20.880 ms ±  4.555 ms  ┊ GC (mean ± σ):  8.28% ± 1.82%

          ▇█
  ▃▁▁▂▁▃▃████▇▆▅▇▅▃▂▂▃▁▃▂▃▃▄▃▂▃▃▃▃▁▁▃▂▁▁▃▃▃▂▃▁▂▂▁▁▁▁▁▂▁▁▁▁▂▁▂ ▃
  16.3 ms         Histogram: frequency by time        32.9 ms <

 Memory estimate: 44.25 MiB, allocs estimate: 6.

Well it's been forever since I looked at this, but my guess is this has something to do with the fact that Julia expects arrays to be column major and Opencv expects them to be row major. When accessing through the ocmat, you're probably hitting non continuous parts of memory. There might be some optimization to be made for Julia arrays that are wrapped in Mat and accessed through Julia functions.

However, this is all just a conjecture and I might be totally wrong.

On Thu, Oct 5, 2023, 4:00 AM Rakesh @.***> wrote:

Some array operations like findall slow and using high memory on OpenCV.Mat compared Array{T, 3}. Here is MWE

using Randomusing BenchmarkToolsusing OpenCV

Random.seed!(1234)

mat = rand(UInt8(0):UInt8(255), 3, 2^10, 2^10) ocmat = OpenCV.Mat(mat)

f = >=(UInt8(100)) @assert findall(f, mat) == findall(f, @. findall(f, mat) == findall(f, @. findall(f, mat) == findall(f, ocmat.data_raw) @benchmark findall($(f), $(mat)) @benchmark findall($(f), $(ocmat)) @benchmark findall($(f), $(ocmat.data)) @benchmark findall($(f), $(ocmat.data_raw))

Results

julia> @benchmark findall($(f), $(mat)) BenchmarkTools.Trial: 269 samples with 1 evaluation. Range (min … max): 15.501 ms … 44.234 ms ┊ GC (min … max): 3.47% … 3.38% Time (median): 17.868 ms ┊ GC (median): 8.16% Time (mean ± σ): 18.622 ms ± 3.149 ms ┊ GC (mean ± σ): 8.28% ± 1.30%
     ██▆▃ ▁
▄▁▁▁▁▁▆██████▅▁▁▁▁▄▁▁▁▁▅▁▁▁▁▁▄▄▅▁▁▄▁▁▁▁▁▁▄▅▁▁▁▁▁▄▁▁▁▁▁▁▁▁▁▆ ▅ 15.5 ms Histogram: log(frequency) by time 31.9 ms <

Memory estimate: 44.25 MiB, allocs estimate: 6.

julia> @benchmark findall($(f), $(ocmat)) BenchmarkTools.Trial: 2 samples with 1 evaluation. Range (min … max): 2.491 s … 2.524 s ┊ GC (min … max): 0.79% … 0.78% Time (median): 2.507 s ┊ GC (median): 0.78% Time (mean ± σ): 2.507 s ± 23.730 ms ┊ GC (mean ± σ): 0.78% ± 0.01%

█ █ █▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁▁█ ▁ 2.49 s Histogram: frequency by time 2.52 s <

Memory estimate: 542.59 MiB, allocs estimate: 26367765.

julia> @benchmark findall($(f), $(ocmat.data)) BenchmarkTools.Trial: 252 samples with 1 evaluation. Range (min … max): 15.296 ms … 46.063 ms ┊ GC (min … max): 2.31% … 6.62% Time (median): 18.633 ms ┊ GC (median): 8.80% Time (mean ± σ): 19.874 ms ± 3.541 ms ┊ GC (mean ± σ): 8.27% ± 1.33%
         ▆█▇▄▄▁
▄▄▁▁▁▁▁▁▁▁▇██████▆█▄▄▆▄▇▁▆▇▄▁▄▄▁▄▁▆▁▁▁▄▁▁▆▄▄▁▄▄▆▇▆▆▄▄▁▁▄▁▄▆ ▆ 15.3 ms Histogram: log(frequency) by time 30 ms <

Memory estimate: 44.25 MiB, allocs estimate: 6.

julia> @benchmark findall($(f), $(ocmat.data_raw)) BenchmarkTools.Trial: 239 samples with 1 evaluation. Range (min … max): 16.272 ms … 62.863 ms ┊ GC (min … max): 9.46% … 2.66% Time (median): 19.361 ms ┊ GC (median): 8.68% Time (mean ± σ): 20.880 ms ± 4.555 ms ┊ GC (mean ± σ): 8.28% ± 1.82%
      ▇█
▃▁▁▂▁▃▃████▇▆▅▇▅▃▂▂▃▁▃▂▃▃▄▃▂▃▃▃▃▁▁▃▂▁▁▃▃▃▂▃▁▂▂▁▁▁▁▁▂▁▁▁▁▂▁▂ ▃ 16.3 ms Histogram: frequency by time 32.9 ms <

Memory estimate: 44.25 MiB, allocs estimate: 6.

— Reply to this email directly, view it on GitHub https://github.com/JuliaImages/OpenCV.jl/issues/32, or unsubscribe https://github.com/notifications/unsubscribe-auth/AA5NR4SVUJKH443SNTT35SLX5YPE5AVCNFSM6AAAAAA5TREVZKVHI2DSMVQWIX3LMV43ASLTON2WKOZRHEZDOMRZGE4DIMQ . You are receiving this because you are subscribed to this thread.Message ID: @.***>

JuliaImages / OpenCV.jl

Some operations on `OpenCV.Mat` are slow and taking high memory. #32

Results