search

JuliaImages / ImageQualityIndexes.jl

Indexes for image quality assessment
MIT License
9 stars 10 forks source link

assess_ssim doesn't work on arrays of more than 2 dimensions #53

Closed ymtoo closed 1 year ago

ymtoo commented 1 year ago

MWE:

(jl_Q3Mnwh) pkg> st
Status `/tmp/jl_Q3Mnwh/Project.toml`
  [2996bd0c] ImageQualityIndexes v0.3.3

julia> using ImageQualityIndexes

julia> assess_ssim(rand(300,300),rand(300,300))
0.005121316421804566

julia> assess_ssim(rand(3,300,300),rand(3,300,300))
ERROR: MethodError: no method matching _imfilter_tiled_swap!(::ComputationalResources.CPU1{ImageFiltering.Algorithm.FIRTiled{3}}, ::Array{Float64, 3}, ::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 2, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}}, ::ImageFiltering.NoPad{ImageFiltering.Pad{3}}, ::Tuple{TiledIteration.TileBuffer{Float64, 3, 3}, OffsetArrays.OffsetArray{Float64, 3, Array{Float64, 3}}})
Closest candidates are:
  _imfilter_tiled_swap!(::Any, ::Any, ::Tuple{Any}, ::Any, ::Tuple{TiledIteration.TileBuffer, Array}) at ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:925
  _imfilter_tiled_swap!(::Any, ::Any, ::Tuple{Any, Any, Vararg{Any}}, ::Any, ::Tuple{TiledIteration.TileBuffer, Array}) at ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:914
Stacktrace:
  [1] _imfilter_tiled_swap!(r::ComputationalResources.CPU1{ImageFiltering.Algorithm.FIRTiled{3}}, out::Array{Float64, 3}, kernel::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 1, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 2, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}}, border::ImageFiltering.NoPad{ImageFiltering.Pad{3}}, tmp::Tuple{TiledIteration.TileBuffer{Float64, 3, 3}, Array{Float64, 3}})
    @ ImageFiltering ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:921
  [2] _imfilter_tiled!(r::ComputationalResources.CPU1{ImageFiltering.Algorithm.FIRTiled{3}}, out::Array{Float64, 3}, A::OffsetArrays.OffsetArray{Float64, 3, Array{Float64, 3}}, kernel::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 0, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 1, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 2, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}}, border::ImageFiltering.NoPad{ImageFiltering.Pad{3}}, tiles::Vector{Tuple{Array{Float64, 3}, Array{Float64, 3}}}, indsout::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}, Base.OneTo{Int64}})
    @ ImageFiltering ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:884
  [3] imfilter!(r::ComputationalResources.CPU1{ImageFiltering.Algorithm.FIRTiled{3}}, out::Array{Float64, 3}, A::OffsetArrays.OffsetArray{Float64, 3, Array{Float64, 3}}, kernel::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 0, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 1, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 2, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}}, border::ImageFiltering.NoPad{ImageFiltering.Pad{3}}, inds::Tuple{Base.OneTo{Int64}, Base.OneTo{Int64}, Base.OneTo{Int64}})
    @ ImageFiltering ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:778
  [4] imfilter!
    @ ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:775 [inlined]
  [5] imfilter!
    @ ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:716 [inlined]
  [6] imfilter!(r::ComputationalResources.CPU1{ImageFiltering.Algorithm.FIRTiled{3}}, out::Array{Float64, 3}, img::Array{Float64, 3}, kernel::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 0, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 1, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 2, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}}, border::ImageFiltering.Pad{0})
    @ ImageFiltering ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:703
  [7] imfilter!(out::Array{Float64, 3}, img::Array{Float64, 3}, kernel::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 0, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 1, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 2, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}}, border::ImageFiltering.Pad{0}, alg::ImageFiltering.Algorithm.FIRTiled{3})
    @ ImageFiltering ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:612
  [8] imfilter!
    @ ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:606 [inlined]
  [9] imfilter
    @ ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:27 [inlined]
 [10] imfilter(::Type{Float64}, ::Array{Float64, 3}, ::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 0, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 1, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 2, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}}, ::String)
    @ ImageFiltering ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:22
 [11] imfilter(img::Array{Float64, 3}, kernel::Tuple{ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 0, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 1, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, ImageFiltering.KernelFactors.ReshapedOneD{Float64, 3, 2, OffsetArrays.OffsetVector{Float64, Vector{Float64}}}}, args::String)
    @ ImageFiltering ~/.julia/packages/ImageFiltering/B0lkl/src/imfilter.jl:5
 [12] invokelatest(::Any, ::Any, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Base ./essentials.jl:729
 [13] invokelatest(::Any, ::Any, ::Vararg{Any})
    @ Base ./essentials.jl:726
 [14] (::LazyModules.LazyFunction)(::Array{Float64, 3}, ::Vararg{Any}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ LazyModules ~/.julia/packages/LazyModules/d9Be6/src/LazyModules.jl:29
 [15] (::LazyModules.LazyFunction)(::Array{Float64, 3}, ::Vararg{Any})
    @ LazyModules ~/.julia/packages/LazyModules/d9Be6/src/LazyModules.jl:27
 [16] _ssim_statistics(x::Array{Float64, 3}, ref::Array{Float64, 3}, window::OffsetArrays.OffsetVector{Float64, Vector{Float64}}; crop::Bool)
    @ ImageQualityIndexes ~/.julia/packages/ImageQualityIndexes/M3pul/src/ssim.jl:184
 [17] __ssim_map_fast(x::Array{Float64, 3}, ref::Array{Float64, 3}, window::OffsetArrays.OffsetVector{Float64, Vector{Float64}}, C₁::Float64, C₂::Float64; crop::Bool)
    @ ImageQualityIndexes ~/.julia/packages/ImageQualityIndexes/M3pul/src/ssim.jl:151
 [18] _ssim_map(iqi::SSIM{OffsetArrays.OffsetVector{Float64, Vector{Float64}}}, x::Array{Float64, 3}, ref::Array{Float64, 3}, peakval::Float64, K::Tuple{Float64, Float64})
    @ ImageQualityIndexes ~/.julia/packages/ImageQualityIndexes/M3pul/src/ssim.jl:122
 [19] _ssim_map
    @ ~/.julia/packages/ImageQualityIndexes/M3pul/src/ssim.jl:105 [inlined]
 [20] (::SSIM{OffsetArrays.OffsetVector{Float64, Vector{Float64}}})(x::Array{Float64, 3}, ref::Array{Float64, 3})
    @ ImageQualityIndexes ~/.julia/packages/ImageQualityIndexes/M3pul/src/ssim.jl:85
 [21] #assess_ssim#5
    @ ~/.julia/packages/ImageQualityIndexes/M3pul/src/ssim.jl:88 [inlined]
 [22] assess_ssim(x::Array{Float64, 3}, ref::Array{Float64, 3})
    @ ImageQualityIndexes ~/.julia/packages/ImageQualityIndexes/M3pul/src/ssim.jl:88
 [23] top-level scope
    @ REPL[7]:1

It works if TiledIteration is downgraded to v0.3.1.

(jl_9viMqr) pkg> st
Status `/tmp/jl_9viMqr/Project.toml`
  [2996bd0c] ImageQualityIndexes v0.3.3
⌃ [06e1c1a7] TiledIteration v0.3.1
Info Packages marked with ⌃ have new versions available and may be upgradable.

julia> using ImageQualityIndexes

julia> assess_ssim(rand(300,300),rand(300,300))
0.005284005212347047

julia> assess_ssim(rand(3,300,300),rand(3,300,300))
0.008212332218443523
johnnychen94 commented 1 year ago

Good catch! It looks like imfilter is broken somewhere. I'm not sure if I had the time to fix this by myself (super busy), I'll try if I can manage out sometime this weekend or next weekend. It would be great if you can submit bugfix patch here or in ImageFiltering.

BTW, if you're measuring SSIM over RGB images, you can do

assess_ssim(colorview(RGB, rand(3,300,300)), colorview(RGB, rand(3, 300, 300)))

CRef: https://dsp.stackexchange.com/a/82766/62632

ymtoo commented 1 year ago

By comparing TiledIteration v0.3.1 and v0.4.0 (https://github.com/JuliaArrays/TiledIteration.jl/compare/v0.3.1...v0.4.0#diff-305b09ea941c21408c23ff438666abdec63b45b275cf1d62a98e1b0085576e7aR270), parent arrays of TileBuffer are not the same. 

Base.parent(tb::TileBuffer) = tb.buf # v0.3.1

Base.parent(tb::TileBuffer) = tb.view # v0.4.0

Is this intended?