XGBoost Broken - won't work after fresh install of Julia 1.8

[compleathorseplayer]

Hi. XGBoost seems to be broken. It failed to precompile in Julia 1.7 so I installed/Julia 1.8 fresh.

julia> using XGBoost
 │ Package XGBoost not found, but a package named XGBoost is available from a registry. 
 │ Install package?
 │   (@v1.8) pkg> add XGBoost 
 └ (y/n/o) [y]: 
   Resolving package versions...
    Updating `~/.julia/environments/v1.8/Project.toml`
  [009559a3] + XGBoost v1.5.2
    Updating `~/.julia/environments/v1.8/Manifest.toml`
  [009559a3] + XGBoost v1.5.2
  [1d63c593] + LLVMOpenMP_jll v14.0.4+0
  [a5c6f535] + XGBoost_jll v1.6.1+0
Precompiling project...
  ✗ XGBoost
  2 dependencies successfully precompiled in 3 seconds. 109 already precompiled.
  1 dependency errored. To see a full report either run `import Pkg; Pkg.precompile()` or load the package
[ Info: Precompiling XGBoost [009559a3-9522-5dbb-924b-0b6ed2b22bb9]
ERROR: LoadError: InitError: could not load library "/Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib"
dlopen(/Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib, 1): Symbol not found: ___cxa_deleted_virtual
  Referenced from: /Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib (which was built for Mac OS X 10.15)
  Expected in: /usr/lib/libc++.1.dylib
 in /Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:116
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ XGBoost_jll ~/.julia/packages/XGBoost_jll/Ytq8S/src/wrappers/x86_64-apple-darwin.jl:10
  [5] _include_from_serialized(pkg::Base.PkgId, path::String, depmods::Vector{Any})
    @ Base ./loading.jl:831
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt64)
    @ Base ./loading.jl:1039
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1315
  [8] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1200
  [9] macro expansion
    @ ./loading.jl:1180 [inlined]
 [10] macro expansion
    @ ./lock.jl:223 [inlined]
 [11] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1144
 [12] include
    @ ./Base.jl:419 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1554
 [14] top-level scope
    @ stdin:1
during initialization of module XGBoost_jll
in expression starting at /Users/davide/.julia/packages/XGBoost/D30Xd/src/XGBoost.jl:1
in expression starting at stdin:1
ERROR: Failed to precompile XGBoost [009559a3-9522-5dbb-924b-0b6ed2b22bb9] to /Users/davide/.julia/compiled/v1.8/XGBoost/jl_yWXGZF.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:1705
 [3] compilecache
   @ ./loading.jl:1649 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1337
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [6] macro expansion
   @ ./loading.jl:1180 [inlined]
 [7] macro expansion
   @ ./lock.jl:223 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [9] top-level scope
   @ ~/.julia/packages/CUDA/DfvRa/src/initialization.jl:52

julia> using XGBoost
[ Info: Precompiling XGBoost [009559a3-9522-5dbb-924b-0b6ed2b22bb9]
ERROR: LoadError: InitError: could not load library "/Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib"
dlopen(/Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib, 1): Symbol not found: ___cxa_deleted_virtual
  Referenced from: /Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib (which was built for Mac OS X 10.15)
  Expected in: /usr/lib/libc++.1.dylib
 in /Users/davide/.julia/artifacts/18efbf9f697f68d4d6b74e269856f743e6ce7d10/lib/libxgboost.dylib
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:116
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ XGBoost_jll ~/.julia/packages/XGBoost_jll/Ytq8S/src/wrappers/x86_64-apple-darwin.jl:10
  [5] _include_from_serialized(pkg::Base.PkgId, path::String, depmods::Vector{Any})
    @ Base ./loading.jl:831
  [6] _require_search_from_serialized(pkg::Base.PkgId, sourcepath::String, build_id::UInt64)
    @ Base ./loading.jl:1039
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1315
  [8] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1200
  [9] macro expansion
    @ ./loading.jl:1180 [inlined]
 [10] macro expansion
    @ ./lock.jl:223 [inlined]
 [11] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1144
 [12] include
    @ ./Base.jl:419 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1554
 [14] top-level scope
    @ stdin:1
during initialization of module XGBoost_jll
in expression starting at /Users/davide/.julia/packages/XGBoost/D30Xd/src/XGBoost.jl:1
in expression starting at stdin:1
ERROR: Failed to precompile XGBoost [009559a3-9522-5dbb-924b-0b6ed2b22bb9] to /Users/davide/.julia/compiled/v1.8/XGBoost/jl_2a45v3.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:1705
 [3] compilecache
   @ ./loading.jl:1649 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1337
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [6] macro expansion
   @ ./loading.jl:1180 [inlined]
 [7] macro expansion
   @ ./lock.jl:223 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
 [9] top-level scope
   @ ~/.julia/packages/CUDA/DfvRa/src/initialization.jl:52

Does anyone have any ideas? Are there any alternative versions of xgboost that work on Julia?

Thanks for any help.

[giordano]

As written in the error message (and explained also on discourse), it's the binary library xgboost which requires macOS 10.15, there's nothing that can be done about that here. But you can use XGBoost_jll@1.5.2.

[hcho3]

An XGBoost project maintainer here.

There's nothing upstream XGBoost that requires MacOS 10.15 specifically. In the Python package, we target MacOS 10.13+: https://github.com/dmlc/xgboost/blob/ada4a86d1c3687ba78392993e6b2902d69175ef2/tests/ci_build/build_python_wheels.sh#L38

Which libxgboost.dylib the Julia package using? Is it using Homebrew's package (https://formulae.brew.sh/formula/xgboost) ?

[giordano]

Building xgboost in https://github.com/JuliaPackaging/Yggdrasil/blob/728e7c9e273188fefbe5ac008949f269fd9c97a3/X/XGBoost/build_tarballs.jl#L22-L30 requires ___cxa_deleted_virtual (referenced by json.cc.o according to https://github.com/JuliaPackaging/Yggdrasil/pull/4932#issuecomment-1134416085), which is in 10.15.

[hcho3]

Thanks for pointing it out. It appears that this line https://github.com/dmlc/xgboost/blob/ada4a86d1c3687ba78392993e6b2902d69175ef2/include/xgboost/json.h#L61

is causing the issue. I can submit a pull request to remove this line. What's the best way to test the patch with older MacOS?

[giordano]

If you can cook up a patch (also here) I can try it over at Yggdrasil. Is just removing that line enough?

[hcho3]

Yes, that should be sufficient. The function operator= is not used anywhere.

[compleathorseplayer]

Great! Thanks all!

[giordano]

This should have been fixed in XGBoost_jll version 1.6.1+1.

[giordano]

BTW, does this line https://github.com/dmlc/xgboost/blob/ada4a86d1c3687ba78392993e6b2902d69175ef2/tests/ci_build/build_python_wheels.sh#L36 mean that you're actually using the 10.15 SDK for building, but targeting 10.13? We could have done the same in Yggdrasil, although simply removing the offending line is a better fix overall.

[hcho3]

you're actually using the 10.15 SDK for building, but targeting 10.13

Yes.