Open TheCedarPrince opened 1 year ago
ashryaagr
commented
1 year ago Hi. It seems that there is no issue on your end. But as pointed out on #78 , this might be a compatibility error. I will try to get this to work. If I can't get this to work, then I will try to send you the Project.toml i.e. exact versions of the setup I had (old list though), which worked.
TheCedarPrince
commented
1 year ago Hi @ashryaagr - ah I thought that it was related! If you don't have time right now, if you send me the Project.toml of the older version I could create a PR that adds compat entries to the Project.toml of Fairness. Then, as you go through trying to see what can be updated with CompatHelper, we could update the compat entries incrementally until things are largely up to date. Would that work/be helpful? Feel free to drop the toml here and I could pick up on things from there.
ashryaagr
commented
1 year ago Hi @TheCedarPrince - Here is the Project.toml . This is basically the same as that currently on repository.
My best guess for the reason for this issue is that MLJFlux has been updated and Fairness.jl is behind on updates (most probably only with compat updates). So, can you try by restricting MLJFLux to be of version v0.1.6. Pkg.add("MLJFlux@v0.1.6").
And once Fairness.jl gets back on track with the updates, it should be usable with the latest MLJ and MLJFlux version
Big thanks for taking the initiative to help with the compat entries!!
ashryaagr
commented
1 year ago Update - turns out that restricting to v1.6 leaves no satisfiable version of Flux to install due to some CUDA issue. It might also be because I opened my dual-booted linux system after a long time and lag on some update.
ashryaagr
commented
1 year ago ### So here is a solution - not a standard solution, but a workaround for the time-being. You can download the repository and delete all the compat requirements from Fairness. Then you can install that package. Then you can install latest version of MLJ and MLJFlux. I have tried this on my system and this fix does work. I do not get any errors with @load NeuralNetworkClassifier
And in case you still run into an error like this (I did, but was able to solve this):
ERROR: LoadError: UndefVarError: @aggressive_constprop not defined
then, delete the packages in your ~/.julia/packages . Then re-install the downloaded repository, MLJ and MLJFlux. This solved this problem and everything ran smoothly.
The only issue is that LinProgWrapper and PenaltyAlgorithm give errors during tests on my system - perhaps because they require changes with these latest packages. But anyways these two algorithms are our own proposed algorithms. The other standard algorithms like reweighing, equalized odds and calibrated equalized odds work perfectly.
TheCedarPrince
commented
1 year ago Hey! Thanks for the temporary workaround! I can confirm I was able to get this working but instead of destroying everything in .julia/packages, I was able to do the following:
pkg> activate @DEV)@DEV> dev /path/to/Fairness/.)@DEV> add MLJ, MLJFlux@DEV> up And that works. I ran the test suite for Fairness and can confirm the only failing tests are for LinProgWrapper and PenaltyWrapper -- that may need to be its own issue perhaps.
I'll open a PR on compat entries that we can start experimenting with. Thanks for all the help so far!
P.S. For completeness, here is the stack trace of the failed tests:
Testing Running tests...
Test Summary: | Pass Total Time
fair_tensor | 3 3 16.3s
Evaluating over 6 folds: 100%[=========================] Time: 0:00:06
Evaluating over 6 folds: 100%[=========================] Time: 0:00:00
Evaluating over 6 folds: 100%[=========================] Time: 0:00:00
Evaluating over 6 folds: 100%[=========================] Time: 0:00:00
Test Summary: | Pass Total Time
measures | 56 56 24.6s
Test Summary: | Pass Total Time
dataset_macros | 24 24 1m10.9s
[ Info: Training machine(ReweighingWrapper(grp = Sex, …), …).
[ Info: Training machine(ConstantClassifier(), …).
[ Info: Training machine(ReweighingSamplingWrapper(grp = Sex, …), …).
[ Info: Training machine(ConstantClassifier(), …).
[ Info: Training machine(EqOddsWrapper(grp = Sex, …), …).
[ Info: Training machine(ConstantClassifier(), …).
[ Info: Training machine(CalEqOddsWrapper(grp = Sex, …), …).
[ Info: Training machine(ConstantClassifier(), …).
[ Info: Training machine(LinProgWrapper(grp = Sex, …), …).
[ Info: Training machine(ConstantClassifier(), …).
┌ Error: Problem fitting the machine machine(LinProgWrapper(grp = Sex, …), …
).
└ @ MLJBase ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:627
[ Info: Running type checks...
[ Info: Type checks okay.
LinProgWrapper Postprocessing: Error During Test at /home/src/Projects/open_
source_contribs/Fairness.jl/test/algorithms/postprocessing.jl:26
Got exception outside of a @test
UndefVarError: add_NL_constraint not defined
Stacktrace:
[1] getproperty
@ ./Base.jl:31 [inlined]
[2] fit(model::LinProgWrapper{ConstantClassifier}, verbosity::Int64, X::
DataFrames.DataFrame, y::CategoricalArrays.CategoricalVector{Int64, UInt32,
Int64, CategoricalArrays.CategoricalValue{Int64, UInt32}, Union{}})
@ Fairness /home/src/Projects/open_source_contribs/Fairness.jl/src/alg
orithms/postprocessing/linprog.jl:106
[3] fit_only!(mach::Machine{LinProgWrapper{ConstantClassifier}, true}; r
ows::Nothing, verbosity::Int64, force::Bool)
@ MLJBase ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:625
[4] fit_only!
@ ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:572 [inlined]
[5] #fit!#61
@ ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:693 [inlined]
[6] fit!(mach::Machine{LinProgWrapper{ConstantClassifier}, true})
@ MLJBase ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:690
[7] macro expansion
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/algorithms/
postprocessing.jl:31 [inlined]
[8] macro expansion
@ /home/src/Programs/Julia/julia-1.8.0/share/julia/stdlib/v1.8/Test/sr
c/Test.jl:1357 [inlined]
[9] top-level scope
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/algorithms/
postprocessing.jl:27
[10] include(fname::String)
@ Base.MainInclude ./client.jl:476
[11] top-level scope
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/algorithms/
algorithms.jl:2
[12] include(fname::String)
@ Base.MainInclude ./client.jl:476
[13] macro expansion
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/runtests.jl
:19 [inlined]
[14] macro expansion
@ /home/src/Programs/Julia/julia-1.8.0/share/julia/stdlib/v1.8/Test/sr
c/Test.jl:1357 [inlined]
[15] top-level scope
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/runtests.jl
:19
[16] include(fname::String)
@ Base.MainInclude ./client.jl:476
[17] top-level scope
@ none:6
[18] eval
@ ./boot.jl:368 [inlined]
[19] exec_options(opts::Base.JLOptions)
@ Base ./client.jl:276
[20] _start()
@ Base ./client.jl:522
┌ Warning: The `@pipeline` macro is deprecated. For pipelines without target
transformations use pipe syntax, as in `ContinuousEncoder() |> Standardizer
() |> my_classifier`. For details, query the `Pipeline` docstring. To wrap a
supervised model in a target transformation, use `TransformedTargetModel`,
as in `TransformedTargetModel(my_regressor, target=Standardizer())`
│ caller = ip:0x0
└ @ Core :-1
[ Info: For silent loading, specify `verbosity=0`.
import MLJDecisionTreeInterface ✔
[ Info: For silent loading, specify `verbosity=0`.
import MLJDecisionTreeInterface ✔
[ Info: Training machine(PenaltyWrapper(grp = sex, …), …).
┌ Error: Problem fitting the machine machine(PenaltyWrapper(grp = sex, …), …
).
└ @ MLJBase ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:627
[ Info: Running type checks...
[ Info: Type checks okay.
PenaltyWrapper Inprocessing: Error During Test at /home/src/Projects/open_so
urce_contribs/Fairness.jl/test/algorithms/inprocessing.jl:1
Got exception outside of a @test
MethodError: no method matching Fairness.FairTensor(::Array{Any, 3}, ::Vec
tor{InlineStrings.String7})
Closest candidates are:
Fairness.FairTensor(::Any, ::Vector{String}) at /home/src/Projects/open_
source_contribs/Fairness.jl/src/fair_tensor.jl:17
Stacktrace:
[1] fit(model::PenaltyWrapper{Pipeline316}, verbosity::Int64, X::DataFra
mes.DataFrame, y::CategoricalArrays.CategoricalVector{Int64, UInt32, Int64,
CategoricalArrays.CategoricalValue{Int64, UInt32}, Union{}})
@ Fairness /home/src/Projects/open_source_contribs/Fairness.jl/src/alg
orithms/inprocessing/penalty.jl:64
[2] fit_only!(mach::Machine{PenaltyWrapper{Pipeline316}, true}; rows::No
thing, verbosity::Int64, force::Bool)
@ MLJBase ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:625
[3] fit_only!
@ ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:572 [inlined]
[4] #fit!#61
@ ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:693 [inlined]
[5] fit!(mach::Machine{PenaltyWrapper{Pipeline316}, true})
@ MLJBase ~/.julia/packages/MLJBase/CtxrQ/src/machines.jl:690
[6] macro expansion
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/algorithms/
inprocessing.jl:6 [inlined]
[7] macro expansion
@ /home/src/Programs/Julia/julia-1.8.0/share/julia/stdlib/v1.8/Test/sr
c/Test.jl:1357 [inlined]
[8] top-level scope
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/algorithms/
inprocessing.jl:2
[9] include(fname::String)
@ Base.MainInclude ./client.jl:476
[10] top-level scope
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/algorithms/
algorithms.jl:3
[11] include(fname::String)
@ Base.MainInclude ./client.jl:476
[12] macro expansion
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/runtests.jl
:19 [inlined]
[13] macro expansion
@ /home/src/Programs/Julia/julia-1.8.0/share/julia/stdlib/v1.8/Test/sr
c/Test.jl:1357 [inlined]
[14] top-level scope
@ /home/src/Projects/open_source_contribs/Fairness.jl/test/runtests.jl
:19
[15] include(fname::String)
@ Base.MainInclude ./client.jl:476
[16] top-level scope
@ none:6
[17] eval
@ ./boot.jl:368 [inlined]
[18] exec_options(opts::Base.JLOptions)
@ Base ./client.jl:276
[19] _start()
@ Base ./client.jl:522
Test Summary: | Pass Error Total Time
algorithms | 8 2 10 33.2s
Reweighing | 2 2 4.8s
ReweighingSampling | 2 2 2.0s
Equalized Odds Postprocessing | 2 2 10.1s
Calibrated Equalized Odds Postprocessing | 2 2 2.1s
LinProgWrapper Postprocessing | 1 1 7.0s
PenaltyWrapper Inprocessing | 1 1 3.1s
ERROR: LoadError: Some tests did not pass: 8 passed, 0 failed, 2 errored, 0
broken.
in expression starting at /home/src/Projects/open_source_contribs/Fairness.j
l/test/runtests.jl:18
ERROR: Package Fairness errored during testing
Hi @ashryaagr and maintainers,
I was trying to follow along with the NextJournal Tutorial for Fairness.jl - great tutorial by the way! - but it seems that there is some sort of error now with MLJFlux. Here are the steps to reproduce what I am seeing:
pkg> activate --tempHere is the full stack trace when I try to load a NeuralNetworkClassifier:
I noticed that my package for MLJ was quite old so I also ran an update via
pkg> upafter this error. After updating, trying to load the NeuralNetworkClassifier results in this error:Additionally, here is my current set-up and version info:
Julia Version: 1.8 [7c232609] Fairness v0.3.2 ⌃ [add582a8] MLJ v0.16.7 [094fc8d1] MLJFlux v0.2.9 Info Packages marked with ⌃ have new versions available
What am I doing incorrectly here? Really looking forward to working with this package - thanks! :smile: