Error: no BLAS/LAPACK library loaded! -- PROPACK.jl

[amontoison]

I recently compiled PROPACK with LBT and a user was unable to use it MKL. PROPACK requires an ILP64 BLAS and I don't understand why switching to another backend is not working. https://github.com/JuliaSmoothOptimizers/PROPACK.jl/issues/54

julia> using PROPACK
julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libopenblas64_.so

julia> tsvd(rand(20, 20))

([-0.22500001381615176; -0.22976088455209862; … ; -0.24443800988345776; -0.2390685450952052;;], [10.121123857038096], [-0.19458837206408175; -0.22006568046133926; … ; -0.17804647580393684; -0.22870075370120846;;], [1.1357215518033364e-36], 11, 11)

julia> using PROPACK
julia> using MKL
julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
├ [ILP64] libmkl_rt.so
└ [ LP64] libmkl_rt.so

julia> tsvd(rand(20, 20))
Error: no BLAS/LAPACK library loaded!
([-4.67282983435e-311; 2.37152099006e-311; … ; -9.8452633572525e-311; 1.8422879014037e-310;;], [6.9149587937626e-310], [-0.10445689753014475; 0.2902031860591114; … ; -0.20390069222252044; -0.32014726145774497;;], [0.0], 11, 11)

[amontoison]

I have exactly the same error with BLIS:

julia> BLAS.lbt_forward("/home/alexis/.julia/artifacts/6908daa58e00fa4a08ae4cab660690ee5b9d4630/lib/libblis.so", clear=true, verbose=true)
Generating forwards to /home/alexis/.julia/artifacts/6908daa58e00fa4a08ae4cab660690ee5b9d4630/lib/libblis.so
 -> Autodetected symbol suffix "64_"
 -> Autodetected interface ILP64 (64-bit)
 -> Autodetected normal complex return style
 -> Autodetected gfortran calling convention
 -> CBLAS not found
Processed 4945 symbols; forwarded 157 symbols with 64-bit interface and mangling to a suffix of "64_"
157

julia> BLAS.get_config()
LinearAlgebra.BLAS.LBTConfig
Libraries: 
└ [ILP64] libblis.so

julia> tsvd(rand(20, 20))
Error: no BLAS/LAPACK library loaded!
([NaN; NaN; … ; NaN; NaN;;], [6.9149587937626e-310], [NaN; NaN; … ; NaN; NaN;;], [0.0], 11, 15)

For information, PROPACK was compiled this build_tarballs.jl.

cc @staticfloat

[staticfloat]

Processed 4945 symbols; forwarded 157 symbols with 64-bit interface and mangling to a suffix of "64_"

It looked for 4945 symbols, it only found 157. Sounds like the BLAS library is quite incomplete, or it's finding the wrong symbol suffix. If libblis.so only provides a subset of the symbols that are needed, you can load it with clear=false so that the symbols that libblis.so does not provide are provided instead by OpenBLAS.

If you want to figure out where the missing function call is coming from, do this:

function debug_missing_function()
    println("Missing BLAS/LAPACK function!")
    display(stacktrace())
end
LinearAlgebra.BLAS.lbt_set_default_func(@cfunction(debug_missing_function, Cvoid, ()))

[amontoison]

For MKL, I checked with nm -D $(MKL_jll.libmkl_rt_path) that the symbols required by PROPACK are in the shared library. I will also check for the BLIS library.

[amontoison]

I can confirm that both MKL and BLIS have the BLAS symbols for PROPACK.

[staticfloat]

It's not just that PROPACK is satisfied; any Julia code that is run that uses BLAS symbols needs to be satisfied as well.

[ViralBShah]

Tried with AppleAccelerate and getting same error. The suggested debugging approach doesn't help in understanding what functions are missing.

Missing BLAS/LAPACK function!
22-element Vector{Base.StackTraces.StackFrame}:
 debug_missing_function() at REPL[1]:3
 lansvd!(jobu::Char, jobv::Char, m::Int64, n::Int64, kmax::Int64, aprod::Ptr{Nothing}, U::Matrix{Float64}, s::Vector{Float64}, bnd::Vector{Float64}, V::Matrix{Float64}, tolin::Float64, work::Vector{Float64}, iwork::Vector{Int64}, doption::Vector{Float64}, ioption::Vector{Int64}, dparm::Ptr{Nothing}, iparm::Vector{Int64}; cwork::Nothing) at wrappers.jl:83
 lansvd! at wrappers.jl:42 [inlined]
 lansvd(jobu::Char, jobv::Char, m::Int64, n::Int64, pff::Ptr{Nothing}, initvec::Vector{Float64}, k::Int64, kmax::Int64, tolin::Float64, dparm::Ptr{Nothing}) at wrappers.jl:255
 #tsvd#9 at PROPACK.jl:69 [inlined]
 tsvd(A::LinearOperators.LinearOperator{Float64, Int64, LinearOperators.var"#5#8"{Matrix{Float64}}, LinearOperators.var"#6#9"{Matrix{Float64}}, LinearOperators.var"#7#10"{Matrix{Float64}}, Vector{Float64}}) at PROPACK.jl:57
 tsvd(A::Matrix{Float64}; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}}) at PROPACK.jl:216
 tsvd(A::Matrix{Float64}) at PROPACK.jl:216
 top-level scope at REPL[8]:1
 eval at boot.jl:370 [inlined]
 ⋮
 kwcall(::Any, ::typeof(REPL.start_repl_backend), backend::REPL.REPLBackend, consumer::Any) at REPL.jl:231
 run_repl(repl::REPL.AbstractREPL, consumer::Any; backend_on_current_task::Bool, backend::Any) at REPL.jl:379
 run_repl(repl::REPL.AbstractREPL, consumer::Any) at REPL.jl:365
 (::Base.var"#1017#1019"{Bool, Bool, Bool})(REPL::Module) at client.jl:421
 #invokelatest#2 at essentials.jl:816 [inlined]
 invokelatest at essentials.jl:813 [inlined]
 run_main_repl(interactive::Bool, quiet::Bool, banner::Bool, history_file::Bool, color_set::Bool) at client.jl:405
 exec_options(opts::Base.JLOptions) at client.jl:322
 _start() at client.jl:522

[ViralBShah]

PROPACK looks pretty straightforward - just links directly to LAPACK and BLAS. The build script looks right as well, so I am only guessing that we probably are missing some BLAS or LAPACK functions in the name mangling.

The only observation I have is that PROPACK.jl uses PROPACK_jll 0.2.1 (which perhaps has an older build recipe) and 0.2.3 is being held back by the package resolver due to issues with LinearOperators.

Also, PROPACK seems to have its own copy of LAPACK 3.0. I'm wondering if that is leading to confusion during linking.

[RalphAS]

This may be fixed by https://github.com/JuliaLinearAlgebra/MKL.jl/pull/140

[ViralBShah]

BTW, this did not get fixed by the PR to MKL.jl.

[ViralBShah]

Works now with Julia 1.10.4 and LBT 5.9:

julia> using MKL

julia> using PROPACK

julia> tsvd(rand(20, 20))
([-0.2060419796110353; -0.2454011505143323; … ; -0.19251920965130948; -0.20989604037851442;;], [10.448019745507464], [-0.2316205727195162; -0.222697646579334; … ; -0.15296995156527565; -0.20513227779552395;;], [4.878769996642799e-38], 11, 11)