search

JuliaStats / Rmath.jl

Archive of functions that emulate R's d-p-q-r functions for probability distributions
Other
16 stars 13 forks source link

Julia 1.5 libc.musl, could not load libRmath-julia.so #70

Closed dehann closed 4 months ago

dehann commented 4 years ago

Looks like Rmath library is missing (or not being linked properly) with the new Julia-1.5.0-beta-1 libc.musl binary. I tried a few things to help find where this is happening but got stuck not knowing the internals here so well yet.

cc @ViralBShah

Julia 1.5 beta 1, libc.musl

/tmp/testrm 
$ julia150b1m -O3
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.0-beta1.0 (2020-05-28)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.5) pkg> activate .
 Activating new environment at `/tmp/testrm/Project.toml`

(testrm) pkg> add Rmath
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
Updating `/tmp/testrm/Project.toml`
  [79098fc4] + Rmath v0.6.1
Updating `/tmp/testrm/Manifest.toml`
  [79098fc4] + Rmath v0.6.1
  [f50d1b31] + Rmath_jll v0.2.2+0
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [b77e0a4c] + InteractiveUtils
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [44cfe95a] + Pkg
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode

(testrm) pkg> test Rmath
    Testing Rmath
Status `/tmp/jl_nnkMEf/Project.toml`
  [79098fc4] Rmath v0.6.1
  [f50d1b31] Rmath_jll v0.2.2+0
  [9a3f8284] Random
  [10745b16] Statistics
  [8dfed614] Test
Status `/tmp/jl_nnkMEf/Manifest.toml`
  [79098fc4] Rmath v0.6.1
  [f50d1b31] Rmath_jll v0.2.2+0
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8ba89e20] Distributed
  [b77e0a4c] InteractiveUtils
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [44cfe95a] Pkg
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
ERROR: LoadError: InitError: could not load library "libRmath-julia.so"
Error loading shared library libRmath-julia.so: No such file or directory
Stacktrace:
 [1] __init__() at /home/dehann/.julia/packages/Rmath/lo1Ao/src/Rmath.jl:37
 [2] _include_from_serialized at ./loading.jl:697
 [3] _require_search_from_serialized at ./loading.jl:782
 [4] _require at ./loading.jl:1007
 [5] require at ./loading.jl:928
 [6] require at ./loading.jl:923
 [7] include at ./client.jl:457
 [8] top-level scope at none:6
during initialization of module Rmath
in expression starting at /home/dehann/.julia/packages/Rmath/lo1Ao/test/runtests.jl:1
ERROR: Package Rmath errored during testing

Same thing happens after I try pkg> build Rmath.

So, I checked for the libRmath library location:

julia-1.5.0-beta1/bin 
$ readelf -d julia 

Dynamic section at offset 0x2df0 contains 22 entries:
  Tag        Type                         Name/Value
 0x0000000000000001 (NEEDED)             Shared library: [libjulia.so.1]
 0x0000000000000001 (NEEDED)             Shared library: [libc.musl-x86_64.so.1]
 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/../lib:$ORIGIN/../lib/julia]
 0x000000000000000c (INIT)               0x401138
 0x000000000000000d (FINI)               0x401bb2
 0x0000000000000019 (INIT_ARRAY)         0x602dc8
 0x000000000000001b (INIT_ARRAYSZ)       8 (bytes)
 0x0000000000000004 (HASH)               0x400258
 0x0000000000000005 (STRTAB)             0x4008e0
 0x0000000000000006 (SYMTAB)             0x4003d0
 0x000000000000000a (STRSZ)              911 (bytes)
 0x000000000000000b (SYMENT)             24 (bytes)
 0x0000000000000015 (DEBUG)              0x0
 0x0000000000000003 (PLTGOT)             0x603000
 0x0000000000000002 (PLTRELSZ)           936 (bytes)
 0x0000000000000014 (PLTREL)             RELA
 0x0000000000000017 (JMPREL)             0x400d90
 0x0000000000000007 (RELA)               0x400c70
 0x0000000000000008 (RELASZ)             288 (bytes)
 0x0000000000000009 (RELAENT)            24 (bytes)
 0x000000006ffffffb (FLAGS_1)            Flags: ORIGIN
 0x0000000000000000 (NULL)               0x0

But cannot find anything resembling librmath in either Julia's provided lib/ or lib/julia/ folder. So I'm a little lost on where that library is actually being included.

Also:

   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.0-beta1.0 (2020-05-28)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

julia> versioninfo()
Julia Version 1.5.0-beta1.0
Commit 6443f6c95a (2020-05-28 17:42 UTC)
Platform Info:
  OS: Linux (x86_64-linux-musl)
  CPU: Intel(R) Core(TM) i7-9750H CPU @ 2.60GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-9.0.1 (ORCJIT, skylake)
Environment:
  JULIA_NUM_THREADS = 6

But still works with glibc version

glibc/julia-1.5.0-beta1/bin 
$ ./julia -O3
               _
   _       _ _(_)_     |  Documentation: https://docs.julialang.org
  (_)     | (_) (_)    |
   _ _   _| |_  __ _   |  Type "?" for help, "]?" for Pkg help.
  | | | | | | |/ _` |  |
  | | |_| | | | (_| |  |  Version 1.5.0-beta1.0 (2020-05-28)
 _/ |\__'_|_|_|\__'_|  |  Official https://julialang.org/ release
|__/                   |

(@v1.5) pkg> add Rmath
   Updating registry at `~/.julia/registries/General`
   Updating git-repo `https://github.com/JuliaRegistries/General.git`
  Resolving package versions...
No Changes to `~/.julia/environments/v1.5/Project.toml`
No Changes to `~/.julia/environments/v1.5/Manifest.toml`

(@v1.5) pkg> test Rmath
    Testing Rmath
Status `/tmp/jl_HDc6j3/Project.toml`
  [79098fc4] Rmath v0.6.1
  [f50d1b31] Rmath_jll v0.2.2+0
  [9a3f8284] Random
  [10745b16] Statistics
  [8dfed614] Test
Status `/tmp/jl_HDc6j3/Manifest.toml`
  [79098fc4] Rmath v0.6.1
  [f50d1b31] Rmath_jll v0.2.2+0
  [2a0f44e3] Base64
  [ade2ca70] Dates
  [8ba89e20] Distributed
  [b77e0a4c] InteractiveUtils
  [76f85450] LibGit2
  [8f399da3] Libdl
  [37e2e46d] LinearAlgebra
  [56ddb016] Logging
  [d6f4376e] Markdown
  [44cfe95a] Pkg
  [de0858da] Printf
  [3fa0cd96] REPL
  [9a3f8284] Random
  [ea8e919c] SHA
  [9e88b42a] Serialization
  [6462fe0b] Sockets
  [2f01184e] SparseArrays
  [10745b16] Statistics
  [8dfed614] Test
  [cf7118a7] UUIDs
  [4ec0a83e] Unicode
    Testing Rmath tests passed

Similarly, still works fine on Julia 1.4.2

Previous related issues

Suspected Issue

This is either an issue at Rmath_jll or here at Rmath.jl, but seems bit strange that musl vs glibc versions have different behaviors?

ViralBShah commented 4 years ago

@giordano is usually the one most familiar with this sort of thing. I am sure this is an Rmath_jll issue.

We stopped shipping libRmath a long time ago as part of Julia. So you have to look for it in .julia/artifacts, IIRC.

ViralBShah commented 4 years ago

Triggering a new build on Yggdrasil - since it is possible the jll was just too old and last built in Nov. The platform list is still the same, but let's see if the new build resolves the issue.

dehann commented 4 years ago

Hi Viral, thanks -- thought I should post since 1.5 is in beta and could maybe help avoid bigger issues later. I tried again here and see that Rmath_jll moved up from v0.2.2+0 to v0.2.2+1, however, the same error message remains unfortunately:

(@1.5) pkg> add Rmath
(@1.5) pkg> build Rmath
(@1.5) pkg> test Rmath

ERROR: LoadError: InitError: could not load library "libRmath-julia.so"
Error loading shared library libRmath-julia.so: No such file or directory
Stacktrace:
 [1] __init__() at /home/dehann/.julia/packages/Rmath/lo1Ao/src/Rmath.jl:37
 [2] _include_from_serialized at ./loading.jl:697
 [3] _require_from_serialized at ./loading.jl:749
 [4] _require at ./loading.jl:1040
 [5] require at ./loading.jl:928
 [6] require at ./loading.jl:923
 [7] include at ./client.jl:457
 [8] top-level scope at none:6
during initialization of module Rmath
in expression starting at /home/dehann/.julia/packages/Rmath/lo1Ao/test/runtests.jl:1
ERROR: Package Rmath errored during testing

I'm not directly using Rmath, but some of the dependencies higher up do which is how I happened upon the issue. I'll try and see if I can learn more and will post if I find anything useful. For example the same error is happening with Gadfly.jl:

(@1.5) pkg> add Gadfly
(@1.5) pkg> test Gadfly

...

ERROR: LoadError: LoadError: InitError: could not load library "libRmath-julia.so"
Error loading shared library libRmath-julia.so: No such file or directory
...
giordano commented 4 years ago

We may have problems with loading some libraries with Musl in general, I can reproduce this issue also with other libraries, but not all of them.

giordano commented 4 years ago

I opened in issue in Julia: https://github.com/JuliaLang/julia/issues/36458 TLDR: when ccalling a function, Julia can't find the library even if it's already open.

dehann commented 4 years ago

Oh great, thanks!

jpsamaroo commented 3 years ago

This is still an issue even on Julia master, where the ccall issue was fixed. It seems like these calls are the culprit: https://github.com/JuliaStats/Rmath.jl/blob/aa1388f63f6d03d2ca31aaa1e69a98d49d3c1d43/src/Rmath.jl#L37-L42

I'll investigate to see if it can be trivially fixed.

jpsamaroo commented 3 years ago

Rmath_jll probably just needs a new Yggdrasil build to use the changes to JLLWrappers. @giordano or @ViralBShah would one of you be able to do this?

jpsamaroo commented 3 years ago

Confirmed, I can fix this and pass tests by changing all references to Rmath_jll.libRmath to Rmath_jll.libRmath_path (which is non-const).

ViralBShah commented 3 years ago

https://github.com/JuliaPackaging/Yggdrasil/pull/2069