JuliaAI / Imbalance.jl

A Julia toolbox with resampling methods to correct for class imbalance.
https://juliaai.github.io/Imbalance.jl/dev/
MIT License
28 stars 1 forks source link

Julia 1.10 breaks Imbalance.jl #73

Closed ablaom closed 10 months ago

ablaom commented 10 months ago

It seems like something in Julia 1.10 has broken Imbalance. The following example from the docstring works fine in Julia 1.9.1 but fails under Julia 1.10.0

@EssamWisam The problem presumably lies with Julia or another package but I wanted to flag this; perhaps you have some time to investigate?

using MLJ
import Imbalance

# set probability of each class
class_probs = [0.5, 0.2, 0.3]
num_rows, num_continuous_feats = 100, 5
# generate a table and categorical vector accordingly
X, y = Imbalance.generate_imbalanced_data(num_rows, num_continuous_feats;
                                  class_probs, rng=42)

# load RandomUndersampler
RandomUndersampler = @load RandomUndersampler pkg=Imbalance

# wrap the model in a machine
undersampler = RandomUndersampler(ratios=Dict(0=>1.0, 1=> 1.0, 2=>1.0),
               rng=42)
mach = machine(undersampler)

# provide the data to transform (there is nothing to fit)
X_under, y_under = transform(mach, X, y)
ERROR: MethodError: no method matching (LittleDict{Union{}, Union{}, KS, VS} where {KS<:Union{…}, VS<:Union{…}})(::Tuple{}, ::Tuple{}) Stacktrace: [1] LittleDict(ks::Tuple{}, vs::Tuple{}) @ OrderedCollections ~/.julia/packages/OrderedCollections/9C4Uz/src/little_dict.jl:54 [2] freeze(dd::LittleDict{CategoricalArrays.CategoricalValue{…}, AbstractVector{…}, Vector{…}, Vector{…}}) @ OrderedCollections ~/.julia/packages/OrderedCollections/9C4Uz/src/little_dict.jl:106 [3] group_inds(categorical_array::CategoricalArrays.CategoricalVector{…}) @ Imbalance ~/.julia/packages/Imbalance/b1Nbf/src/common/utils.jl:38 [4] generic_undersample(::Matrix{…}, ::CategoricalArrays.CategoricalVector{…}, ::Function; ratios::Dict{…}, pass_inds::Bool, is_transposed::Bool, kwargs::@Kwargs{…}) @ Imbalance ~/.julia/packages/Imbalance/b1Nbf/src/generic_resample.jl:98 [5] generic_undersample @ Imbalance ~/.julia/packages/Imbalance/b1Nbf/src/generic_resample.jl:85 [inlined] [6] #random_undersample#106 @ Imbalance ~/.julia/packages/Imbalance/b1Nbf/src/undersampling_methods/random_undersample/random_undersample.jl:136 [inlined] [7] tablify(matrix_func::typeof(Imbalance.random_undersample), X::@NamedTuple{…}, y::CategoricalArrays.CategoricalVector{…}; try_preserve_type::Bool, encode_func::typeof(Imbalance.generic_encoder), decode_func::typeof(Imbalance.generic_decoder), kwargs::@Kwargs{…}) @ Imbalance ~/.julia/packages/Imbalance/b1Nbf/src/table_wrappers.jl:73 [8] tablify @ Imbalance ~/.julia/packages/Imbalance/b1Nbf/src/table_wrappers.jl:57 [inlined] [9] #random_undersample#107 @ Imbalance ~/.julia/packages/Imbalance/b1Nbf/src/undersampling_methods/random_undersample/random_undersample.jl:148 [inlined] [10] transform(r::Imbalance.MLJ.RandomUndersampler{…}, ::Nothing, X::@NamedTuple{…}, y::CategoricalArrays.CategoricalVector{…}) @ Imbalance.MLJ ~/.julia/packages/Imbalance/b1Nbf/src/undersampling_methods/random_undersample/interface_mlj.jl:26 [11] transform(mach::Machine{…}, Xraw::@NamedTuple{…}, Xraw_more::CategoricalArrays.CategoricalVector{…}) @ MLJBase ~/.julia/packages/MLJBase/fEiP2/src/operations.jl:143 [12] top-level scope @ REPL[12]:1 Some type information was truncated. Use `show(err)` to see complete types. Status `/private/var/folders/4n/gvbmlhdc8xj973001s6vdyw00000gq/T/jl_6lpe6U/Project.toml` [c709b415] Imbalance v0.1.2 [add582a8] MLJ v0.20.2 julia> versioninfo() Julia Version 1.10.0 Commit 3120989f39b (2023-12-25 18:01 UTC) Build Info: Official https://julialang.org/ release Platform Info: OS: macOS (x86_64-apple-darwin22.4.0) CPU: 12 × Intel(R) Core(TM) i7-8850H CPU @ 2.60GHz WORD_SIZE: 64 LIBM: libopenlibm LLVM: libLLVM-15.0.7 (ORCJIT, skylake) Threads: 17 on 12 virtual cores Environment: JULIA_LTS_PATH = /Applications/Julia-1.6.app/Contents/Resources/julia/bin/julia JULIA_PATH = /Applications/Julia-1.10.app/Contents/Resources/julia/bin/julia JULIA_EGLOT_PATH = /Applications/Julia-1.10.app/Contents/Resources/julia/bin/julia JULIA_NUM_THREADS = 12 JULIA_NIGHTLY_PATH = /Applications/Julia-1.10.app/Contents/Resources/julia/bin/julia
EssamWisam commented 10 months ago

Have looked into the issue. The way freeze was used in group_inds was a no-op. Probably thought it was freeze! and it was somehow causing the issue. Reordered it to actually freeze the dict after finishing mutation and the issue was solved.

Currently in the process of trigerring a new release.