JuliaStats / Statistics.jl

The Statistics stdlib that ships with Julia.
https://juliastats.org/Statistics.jl/dev/
Other
70 stars 40 forks source link

Inconsistencies in cor calculation #63

Open bkamins opened 3 years ago

bkamins commented 3 years ago

Here are some cor corner cases that would be good to fix by 2.0 release of Julia:

julia> cor(Union{Float64, Missing}[missing]) # this is the expected result
1.0

julia> cor([missing])
missing

julia> cor(Any[missing])
ERROR: MethodError: no method matching zero(::Type{Any})

julia> cor(Any[1])
ERROR: MethodError: no method matching zero(::Type{Any})

EDIT: this is based on the assumption that if someone wants cor and has missing this missing represents a number (and not e.g. a string)

PGS62 commented 7 months ago

Here is another edge case (with Statistics1,10.0 and Julia 1.10)

julia> cor(fill(missing,2,2))
ERROR: MethodError: no method matching copy(::Missing)

julia> cor(fill(missing,3,3))
ERROR: MethodError: no method matching copy(::Missing)

julia> cor(fill(missing,4,4)) #this is the expected result
4×4 Matrix{Missing}:
 missing  missing  missing  missing
 missing  missing  missing  missing
 missing  missing  missing  missing
 missing  missing  missing  missing
mbauman commented 2 weeks ago
julia> cor(fill(missing,4,4)) #this is the expected result
4×4 Matrix{Missing}:
 missing  missing  missing  missing
 missing  missing  missing  missing
 missing  missing  missing  missing
 missing  missing  missing  missing

Is that the expected result? Or should it be akin to the NaN:

julia> cor(fill(NaN,4,4))
4×4 Matrix{Float64}:
   1.0  NaN    NaN    NaN
 NaN      1.0  NaN    NaN
 NaN    NaN      1.0  NaN
 NaN    NaN    NaN      1.0