JuliaStats / Loess.jl

Local regression, so smooooth!
Other
103 stars 34 forks source link

Loess crashes with StackOverflowError when missing values are present #60

Closed dlakelan closed 2 months ago

dlakelan commented 2 years ago
julia> loess([1,2,3],[4,5,6])
Loess.LoessModel{Float64}([1.0; 2.0; 3.0;;], [4.0, 5.0, 6.0], [2.5714285714285725 1.6428571428571432 -0.21428571428571436; 1.741935483870967 2.0483870967741926 -0.20967741935483827; 2.5714285714285716 1.642857142857142 -0.2142857142857134; 0.23809523809523817 0.47619047619047633 0.9523809523809526], Dict([1.5] => 1, [3.0] => 2, [1.0] => 3, [2.0] => 4), Loess.KDTree{Float64}([1.0; 2.0; 3.0;;], [1, 2, 3], Loess.KDInternalNode{Float64}(1, 2.0, Loess.KDInternalNode{Float64}(1, 1.5, Loess.KDLeafNode(), Loess.KDLeafNode()), Loess.KDLeafNode()), Set([[1.5], [3.0], [1.0], [2.0]]), [1.0; 3.0;;]))

julia> loess([missing,1,2,3],[4,missing,5,6])
ERROR: StackOverflowError:
Stacktrace:
     [1] loess(xs::Vector{Union{Missing, Float64}}, ys::Vector{Union{Missing, Float64}})
       @ Loess ~/.julia/packages/Loess/dv7Ad/src/Loess.jl:119
     [2] loess(xs::Vector{Union{Missing, Float64}}, ys::Vector{Union{Missing, Float64}}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
       @ Loess ~/.julia/packages/Loess/dv7Ad/src/Loess.jl:121--- the last 2 lines are repeated 39990 more times ---

This appears to be due to trying to convert the values and then call loess recursively on the new type.

Should probably either throw an error, or have an "ignore missing" option which just drops the x,y values where either the x or the y or both are missing.

mreichMPI-BGC commented 1 year ago

Yes, have the same issue, cf. also https://discourse.julialang.org/t/loess-with-missing-values/90921

dlakelan commented 5 months ago

Note that it seems to occur even if the vectors have no missing values, so long as their type is something like Vector{Union{Missing,Float64}} so stripping missings manually isn't enough, you also need to convert the result to Vector{Float64}