Issues linking MKL pardiso on Linux

[ranjanan]

@evetc noticed that the following code doesn't work, on a fresh install of Julia v1.6.1 on Ubuntu:

using Pardiso, SparseArrays

ps = MKLPardisoSolver()

A = sparse(rand(10, 10))
B = rand(10, 2)
X = zeros(10, 2)
solve!(ps, X, A, B)

Error:

ERROR: could not load library "libmkl_rt"
libmkl_rt.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
   @ Base.Libc.Libdl ./libdl.jl:114
 [2] dlopen (repeats 2 times)
   @ ./libdl.jl:114 [inlined]
 [3] top-level scope
   @ REPL[19]:1

Julia info:

Julia Version 1.6.1
Commit 6aaedecc44 (2021-04-23 05:59 UTC)
Platform Info:
  OS: Linux (x86_64-pc-linux-gnu)
  CPU: Intel(R) Xeon(R) CPU E7-8867 v3 @ 2.50GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-11.0.1 (ORCJIT, haswell)

From https://github.com/Circuitscape/Circuitscape.jl/issues/302#issuecomment-875668004, it seems like Pardiso.MKL_jll.libmkl_rt is pointing to the right place and the library seems to be loading. It's just unclear why Pardiso doesn't seem to see that.

[KristofferC]

I think the issue is that MKL_jll has a lazy artifact and we fail to download it. Should be pretty easy to fix.

[ranjanan]

Just another data point: @evetc just deleted their .julia and reinstalled everything and it started to work.

lazy artifact and we fail to download it

It looks like the right thing is downloaded but it isn't linked properly?

[KristofferC]

julia> ccall((:mkl_domain_get_max_threads, "libmkl_rt"), Int32, (Ptr{Int32},), Ref(Int32(4)))
ERROR: could not load library "libmkl_rt"
libmkl_rt.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] top-level scope
   @ ./REPL[9]:1

julia> libmkl_rt
"/home/kc/.julia/artifacts/a8e009985328801a84c9af6610f94f77a7c12852/lib/libmkl_rt.so"

julia> ccall((:mkl_domain_get_max_threads,  libmkl_rt), Int32, (Ptr{Int32},), Ref(Int32(4)))
8

[KristofferC]

Can you try with https://github.com/JuliaSparse/Pardiso.jl/pull/78.

[ranjanan]

The original poster just deleted their .julia, reinstalled and can't see this issue anymore. I can't see this anymore too. We may have missed the opportunity to see if your PR fixes it.

[KristofferC]

I also got the same issue locally though. But I know it used to work...

[ranjanan]

Right. Maybe we should just find another case of this not working and pull changes. I'll try on antarctic.

[ranjanan]

https://github.com/JuliaSparse/Pardiso.jl/pull/78#issuecomment-876695526

[lucasbanting]

I am getting this issue again:

ERROR: LoadError: InitError: could not load library "/home/bantingl/.julia/artifacts/347e4bf25d69805922225ce6bf819ef0b8715426/lib/libmkl_rt.so"
/home/bantingl/.julia/artifacts/347e4bf25d69805922225ce6bf819ef0b8715426/lib/libmkl_rt.so: cannot open shared object file: No such file or directory

When looking at the artifacts folder:

ls /home/bantingl/.julia/artifacts/347e4bf25d69805922225ce6bf819ef0b8715426/lib/
libmkl_avx2.so.2                  libmkl_core.so           libmkl_intel_thread.so       libmkl_tbb_thread.so
libmkl_avx512.so.2                libmkl_core.so.2         libmkl_intel_thread.so.2     libmkl_tbb_thread.so.2
libmkl_avx.so.2                   libmkl_def.so.2          libmkl_mc3.so.2              libmkl_vml_avx2.so.2
libmkl_blacs_intelmpi_ilp64.so    libmkl_gf_ilp64.so       libmkl_mc.so.2               libmkl_vml_avx512.so.2
libmkl_blacs_intelmpi_ilp64.so.2  libmkl_gf_ilp64.so.2     libmkl_pgi_thread.so         libmkl_vml_avx.so.2
libmkl_blacs_intelmpi_lp64.so     libmkl_gf_lp64.so        libmkl_pgi_thread.so.2       libmkl_vml_cmpt.so.2
libmkl_blacs_intelmpi_lp64.so.2   libmkl_gf_lp64.so.2      libmkl_rt.so                 libmkl_vml_def.so.2
libmkl_blacs_openmpi_ilp64.so     libmkl_gnu_thread.so     libmkl_scalapack_ilp64.so    libmkl_vml_mc2.so.2
libmkl_blacs_openmpi_ilp64.so.2   libmkl_gnu_thread.so.2   libmkl_scalapack_ilp64.so.2  libmkl_vml_mc3.so.2
libmkl_blacs_openmpi_lp64.so      libmkl_intel_ilp64.so    libmkl_scalapack_lp64.so     libmkl_vml_mc.so.2
libmkl_blacs_openmpi_lp64.so.2    libmkl_intel_ilp64.so.2  libmkl_scalapack_lp64.so.2   mkl_msg.cat
libmkl_cdft_core.so               libmkl_intel_lp64.so     libmkl_sequential.so
libmkl_cdft_core.so.2             libmkl_intel_lp64.so.2   libmkl_sequential.so.2

libmkl_rt.so.2 is missing, (libmkl_rt.so is just a symbolic link to libmkl_rt.so.2).

I am on RockyLinux 9.1, not sure if there is a support issue with MKL 2022 or not.

[KristofferC]

Hm, when I download the url from https://github.com/JuliaBinaryWrappers/MKL_jll.jl/blob/e48189829f836e7867e5ca645a1c8f7c0f180e26/Artifacts.toml#LL10C1-L10C1 (which is https://github.com/JuliaBinaryWrappers/MKL_jll.jl/releases/download/MKL-v2022.2.0+0/MKL.v2022.2.0.x86_64-linux-gnu.tar.gz) it does have libmkl_rt.so.2. Could you try delete that folder (artifacts/347e4bf25d69805922225ce6bf819ef0b8715426) and reinstantiate to have the artifact getting re-downloaded?

[lucasbanting]

That allows my code to precompile correctly, but it stall crashes at runtime with:

ERROR: could not load library "libmkl_rt"
libmkl_rt.so: cannot open shared object file: No such file or directory
Stacktrace:
 [1] set_nprocs!
   @ ~/.julia/packages/Pardiso/GDj1P/src/mkl_pardiso.jl:37 [inlined]
 [2] newmark_beta(T::SparseMatrixCSC{Float64, Int64}, R::SparseMatrixCSC{Float64, Int64}, S::SparseMatrixCSC{Float64, Int64}, spatial_function::Matrix{Float64}, time_function::typeof(voltage_function), times::StepRangeLen{Float64, Base.TwicePrecision{Float64}, Base.TwicePrecision{Float64}, Int64}; u0::Nothing, du0::Nothing, Beta::Float64, save_function::typeof(save_function), scale_factor::Float64, print_info::Bool)

[KristofferC]

Can you show what Pardiso.libmkl_rt is? Also Pardiso.LOCAL_MKL_FOUND

[lucasbanting]

Can you show what Pardiso.libmkl_rt is? Also Pardiso.LOCAL_MKL_FOUND

Pardiso.libmkl_rt: "libmkl_rt" Pardiso.LOCAL_MKL_FOUND: false

[KristofferC]

I'm surprised about that because:

https://github.com/JuliaSparse/Pardiso.jl/blob/07d7838665f1649f00fdefab3e7182b1495ad6e8/src/Pardiso.jl#L112-L114

should set it as

julia> Pardiso.libmkl_rt
Base.RefValue{String}("/home/kristofferc/.julia/artifacts/347e4bf25d69805922225ce6bf819ef0b8715426/lib/libmkl_rt.so")

What does Pardiso.MKL_jll.libmkl_rt_path give?

[lucasbanting]

I just restarted my machine and...

display(Pardiso.MKL_jll.libmkl_rt_path) display(Pardiso.libmkl_rt) display(Pardiso.LOCAL_MKL_FOUND)

gives:

"/home/bantingl/.julia/artifacts/347e4bf25d69805922225ce6bf819ef0b8715426/lib/libmkl_rt.so"
Base.RefValue{String}("/home/bantingl/.julia/artifacts/347e4bf25d69805922225ce6bf819ef0b8715426/lib/libmkl_rt.so")
false

Thanks for your time.