search

JuliaGeo / NetCDF.jl

NetCDF support for the julia programming language
http://juliageo.org/NetCDF.jl/
MIT License
115 stars 28 forks source link

libcurl.so.4: version "CURL_4" not found #164

Closed xanfus closed 2 years ago

xanfus commented 2 years ago 
versioninfo()

Julia Version 1.8.1
Commit afb6c60d69* (2022-09-06 15:09 UTC)
Platform Info:
  OS: Linux (x86_64-linux-gnu)
  CPU: 48 × Intel(R) Xeon(R) CPU E5-2695 v2 @ 2.40GHz
  WORD_SIZE: 64
  LIBM: libopenlibm
  LLVM: libLLVM-13.0.1 (ORCJIT, ivybridge)
  Threads: 1 on 48 virtual cores
Environment:
  LD_LIBRARY_PATH = /home/<user>/root/lib:/home/<user>/geant4-install/lib:/home/<user>/root/lib:/home/<user>/geant4-install/lib:/usr/lib/x86_64-linux-gnu:/usr/lib/x86_64-linux-gnu

(@v1.8) pkg> status
Status `~/.julia/environments/v1.8/Project.toml`
  [30363a11] NetCDF v0.11.7

julia> using NetCDF
[ Info: Precompiling NetCDF [30363a11-5582-574a-97bb-aa9a979735b9]
ERROR: LoadError: InitError: could not load library "/home/<user>/.julia/artifacts/461703969206dd426cc6b4d99f69f6ffab2a9779/lib/libnetcdf.so"
/usr/lib/x86_64-linux-gnu/libcurl.so.4: version `CURL_4' not found (required by /home/<user>/.julia/artifacts/461703969206dd426cc6b4d99f69f6ffab2a9779/lib/libnetcdf.so)
Stacktrace:
  [1] dlopen(s::String, flags::UInt32; throw_error::Bool)
    @ Base.Libc.Libdl ./libdl.jl:117
  [2] dlopen(s::String, flags::UInt32)
    @ Base.Libc.Libdl ./libdl.jl:116
  [3] macro expansion
    @ ~/.julia/packages/JLLWrappers/QpMQW/src/products/library_generators.jl:54 [inlined]
  [4] __init__()
    @ NetCDF_jll ~/.julia/packages/NetCDF_jll/BYHmI/src/wrappers/x86_64-linux-gnu.jl:12
  [5] _include_from_serialized(pkg::Base.PkgId, path::String, depmods::Vector{Any})
    @ Base ./loading.jl:831
  [6] _tryrequire_from_serialized(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:978
  [7] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1347
  [8] _require_prelocked(uuidkey::Base.PkgId)
    @ Base ./loading.jl:1200
  [9] macro expansion
    @ ./loading.jl:1180 [inlined]
 [10] macro expansion
    @ ./lock.jl:223 [inlined]
 [11] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:1144
 [12] include
    @ ./Base.jl:419 [inlined]
 [13] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1554
 [14] top-level scope
    @ stdin:1
during initialization of module NetCDF_jll
in expression starting at /home/<user>/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:1
in expression starting at stdin:1
ERROR: Failed to precompile NetCDF [30363a11-5582-574a-97bb-aa9a979735b9] to /home/<user>/.julia/compiled/v1.8/NetCDF/jl_zPUv4A.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:35
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::IO, internal_stdout::IO, keep_loaded_modules::Bool)
   @ Base ./loading.jl:1705
 [3] compilecache
   @ ./loading.jl:1649 [inlined]
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1337
 [5] _require_prelocked(uuidkey::Base.PkgId)
   @ Base ./loading.jl:1200
 [6] macro expansion
   @ ./loading.jl:1180 [inlined]
 [7] macro expansion
   @ ./lock.jl:223 [inlined]
 [8] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:1144
xanfus commented 2 years ago

I'm able to execute "using NetCDF" with NetCDF@0.11.0.

visr commented 2 years ago

I see you have set up your LD_LIBRARY_PATH to look for dynamic libraries first in /usr/lib/x86_64-linux-gnu. There it is loading a libcurl that is apparently incompatible with libnetcdf. If you start julia without that environment variable set I think it would work.

xanfus commented 2 years ago

I've set LD_LIBRARY_PATH to "" by startup.jl, removed NetCDF from /packages, removed /artifacts and /compiled folders. I've restarted julia, added NetCDF, NetCDF had built successfully for the first time, but "using NetCDF" had errored, blaming libcurl in same way as before. NetCDF package installation procedure doesn't load LibCurl or LibCurl_jll packages, yet these are in registries. libcurl isn't among artifacts.

xanfus commented 2 years ago 
uname -a
Linux alpha-Flagman-WX240T4-3-008LH 5.15.0-46-generic #49~20.04.1-Ubuntu SMP Thu Aug 4 19:15:44 UTC 2022 x86_64 x86_64 x86_64 GNU/Linux

lsb_release -a
No LSB modules are available.
Distributor ID: Ubuntu
Description:    Ubuntu 20.04.5 LTS
Release:    20.04
Codename:   focal

I reflect on my guest OS Ubuntu 22.04.1 at another PC, which has "/usr/lib/x86_64-linux-gnu" at the last position in path and has no issues with NetCDF.

xanfus commented 2 years ago

I've cleared ./julia and copied startup.jl back there.

add NetCDF
    Updating registry at `~/.julia/registries/General.toml`
   Resolving package versions...
   Installed Preferences ── v1.3.0
   Installed OpenSSL_jll ── v1.1.17+0
   Installed Libiconv_jll ─ v1.16.1+1
   Installed Adapt ──────── v3.4.0
   Installed OffsetArrays ─ v1.12.7
   Installed XML2_jll ───── v2.9.14+0
   Installed NetCDF_jll ─── v400.902.5+1
   Installed HDF5_jll ───── v1.12.2+2
   Installed Formatting ─── v0.4.2
   Installed NetCDF ─────── v0.11.7
   Installed JLLWrappers ── v1.4.1
   Installed DiskArrays ─── v0.3.6
  Downloaded artifact: XML2
  Downloaded artifact: OpenSSL
  Downloaded artifact: Libiconv
  Downloaded artifact: NetCDF
  Downloaded artifact: HDF5
    Updating `~/.julia/environments/v1.8/Project.toml`
  [30363a11] + NetCDF v0.11.7
    Updating `~/.julia/environments/v1.8/Manifest.toml`
  [79e6a3ab] + Adapt v3.4.0
  [3c3547ce] + DiskArrays v0.3.6
  [59287772] + Formatting v0.4.2
  [692b3bcd] + JLLWrappers v1.4.1
  [30363a11] + NetCDF v0.11.7
  [6fe1bfb0] + OffsetArrays v1.12.7
  [21216c6a] + Preferences v1.3.0
  [0234f1f7] + HDF5_jll v1.12.2+2
  [94ce4f54] + Libiconv_jll v1.16.1+1
  [7243133f] + NetCDF_jll v400.902.5+1
  [458c3c95] + OpenSSL_jll v1.1.17+0
  [02c8fc9c] + XML2_jll v2.9.14+0
  [0dad84c5] + ArgTools v1.1.1
  [56f22d72] + Artifacts
  [2a0f44e3] + Base64
  [ade2ca70] + Dates
  [f43a241f] + Downloads v1.6.0
  [7b1f6079] + FileWatching
  [b77e0a4c] + InteractiveUtils
  [b27032c2] + LibCURL v0.6.3
  [76f85450] + LibGit2
  [8f399da3] + Libdl
  [37e2e46d] + LinearAlgebra
  [56ddb016] + Logging
  [d6f4376e] + Markdown
  [ca575930] + NetworkOptions v1.2.0
  [44cfe95a] + Pkg v1.8.0
  [de0858da] + Printf
  [3fa0cd96] + REPL
  [9a3f8284] + Random
  [ea8e919c] + SHA v0.7.0
  [9e88b42a] + Serialization
  [6462fe0b] + Sockets
  [fa267f1f] + TOML v1.0.0
  [a4e569a6] + Tar v1.10.0
  [cf7118a7] + UUIDs
  [4ec0a83e] + Unicode
  [e66e0078] + CompilerSupportLibraries_jll v0.5.2+0
  [deac9b47] + LibCURL_jll v7.84.0+0
  [29816b5a] + LibSSH2_jll v1.10.2+0
  [c8ffd9c3] + MbedTLS_jll v2.28.0+0
  [14a3606d] + MozillaCACerts_jll v2022.2.1
  [4536629a] + OpenBLAS_jll v0.3.20+0
  [83775a58] + Zlib_jll v1.2.12+3
  [8e850b90] + libblastrampoline_jll v5.1.1+0
  [8e850ede] + nghttp2_jll v1.48.0+0
  [3f19e933] + p7zip_jll v17.4.0+0
Precompiling project...
  18 dependencies successfully precompiled in 5 seconds

Again, NetCDF looks into system folder.

xanfus commented 2 years ago

/<julia>/usr/lib contains libcurl@4.8.0, while /usr/lib/x86_64-linux-gnu contains libcurl@4.6.0. I have to prepend or append (I'll dive into it) the env var.

xanfus commented 2 years ago

Fact is, using NetCDF errors for the first time since julia starts. The next time the line executes normally and functions are loaded into Main. I'll try to go with it.

visr commented 2 years ago

Could it be that removing the env variable in startup.jl is too late? What if you clear it in the shell before starting julia?

xanfus commented 2 years ago

Yes. i've exported empty LD_LIBRARY_PATH in shell, cleared config.jl, removed NetCDF through package manager. julia knows the envvar as "". I've then removed /artifacts, /compiled, NetCDF and NetCDF_jll from /packages. I've added NetCDF via package manager. For the first time, package, which depended on NetCDF, has compiled with no issues. using NetCDF executed with no issues.

But i have to prepare bash for vscode.

xanfus commented 2 years ago

149

LD_LIBRARY_PATH affects performance of NetCDF. Symbols couldn't be loaded in julia, executed under yet unprepared vscode's bash, while julia in bash with empty LD_LIBRARY_PATH executes ncinfo with no issues. Error message has changed slightly since #149:

julia> ncinfo(efp)
ERROR: could not load symbol "nc_open":
./julia: undefined symbol: nc_open
Stacktrace:
 [1] nc_open(ath::String, mode::UInt16, cidp::Vector{Int32})
   @ NetCDF ~/.julia/packages/NetCDF/7hOe9/src/netcdf_c.jl:55
 [2] nc_open
   @ ~/.julia/packages/NetCDF/7hOe9/src/netcdf_helpers.jl:103 [inlined]
 [3] open(fil::String; mode::UInt16, readdimvar::Bool, add_finalizer::Bool)
   @ NetCDF ~/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:913
 [4] open(f::typeof(show), args::String; kwargs::Base.Pairs{Symbol, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
   @ NetCDF ~/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:999
 [5] open
   @ ~/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:998 [inlined]
 [6] ncinfo(fil::String)
   @ NetCDF ~/.julia/packages/NetCDF/7hOe9/src/NetCDF.jl:1101
 [7] top-level scope
   @ REPL[5]:1
visr commented 2 years ago

After changing LD_LIBRARY_PATH you shouldn't have to reinstall anything to see the effect. But thanks for confirming that the issue disappears after unsetting LD_LIBRARY_PATH. With that I believe this issue can be closed. Using it the way you do indeed breaks this package, but that is only because it is doing exactly what it is supposed to be doing. I don't think there is anything we can do here to stop that. I'm not sure what you mean with preparing bash for vscode but try to find a solution that doesn't need LD_LIBRARY_PATH.