search

GenericMappingTools / GMT.jl

Generic Mapping Tools Library Wrapper for Julia
Other
196 stars 28 forks source link

could not load library "libgmt" #298

Closed ZMAlt closed 3 years ago

ZMAlt commented 5 years ago 
julia> using Libdl
julia> push!(Libdl.DL_LOAD_PATH, "/Applications/GMT-6.0.0rc4.app/Contents/Resources/lib")
3-element Array{String,1}:
 "@loader_path/julia"                                   
 "@loader_path"                                         
 "/Applications/GMT-6.0.0rc4.app/Contents/Resources/lib"

julia> using GMT

julia> coast(region=:global, proj=:Winkel, frame=:g, area=10000,
             land=:burlywood4, water=:wheat1, figsize=12, show=true)
ERROR: error compiling finish_PS_module: error compiling gmt: could not load library "libgmt"
dlopen(/Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libgmt.dylib, 1): Library not loaded: @executable_path/../lib/libcurl.4.dylib
  Referenced from: /Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libgmt.dylib
  Reason: Incompatible library version: libgmt.dylib requires version 10.0.0 or later, but libcurl.4.dylib provides version 7.0.0
Stacktrace:
 [1] #coast#193(::Nothing, ::Bool, ::Base.Iterators.Pairs{Symbol,Any,NTuple{8,Symbol},NamedTuple{(:region, :proj, :frame, :area, :land, :water, :figsize, :show),Tuple{Symbol,Symbol,Symbol,Int64,Symbol,Symbol,Int64,Bool}}}, ::Function, ::String) at /Users/zm/.julia/packages/GMT/Ba3R6/src/pscoast.jl:147
 [2] (::getfield(GMT, Symbol("#kw##coast")))(::NamedTuple{(:region, :proj, :frame, :area, :land, :water, :figsize, :show),Tuple{Symbol,Symbol,Symbol,Int64,Symbol,Symbol,Int64,Bool}}, ::typeof(coast)) at ./none:0
 [3] top-level scope at none:0

MacOS gmt6 rc4

joa-quim commented 5 years ago

Hi, Does your GMT installation works outside of Juia? I mean, from the command line. How did you install GMT?

I'm asking this because from the error message the problem seems to be that there is dependency conflict between the libcurl expected by GMT (requires version 10.0.0 or later) and that firstly found in your system (but libcurl.4.dylib provides version 7.0.0)

ZMAlt commented 5 years ago

Hi. I download gmt DMG from Github and add it to my PATH. Is there any problem ?

export PATH="/Applications/GMT-6.0.0rc4.app/Contents/Resources/bin/:$PATH"
joa-quim commented 5 years ago

No, it should be fine.

@PaulWessel, @seisman Any idea where this libcurl dependency conflict is coming from in the GMT dmg?

joa-quim commented 5 years ago

But you didn’t answer. Does it work when you run gmt from the bash shell?

Sent from my iDedo

No dia 17/09/2019, às 02:23, Mao Zhou notifications@github.com escreveu:

Hi. I download gmt DMG from Github and add it to my PATH. Is there any problem ?

export PATH="/Applications/GMT-6.0.0rc4.app/Contents/Resources/bin/:$PATH" — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

ZMAlt commented 5 years ago

of course

~ $ gmt
    GMT - The Generic Mapping Tools, Version 6.0.0rc4 [64-bit] [4 cores]
    (c) 1991-2019 The GMT Team (https://www.generic-mapping-tools.org/team.html).

Supported in part by the US National Science Foundation (http://www.nsf.gov/)
and volunteers from around the world.

GMT is distributed under the GNU LGP License (http://www.gnu.org/licenses/lgpl.html).

usage: gmt [options]
       gmt <module name> [<module-options>]

options:
  --help            List descriptions of available GMT modules.
  --new-script      Write GMT modern mode script template to stdout.
  --show-bindir     Show directory with GMT executables.
  --show-citation   Show the most recent citation for GMT.
  --show-cores      Show number of available cores.
  --show-datadir    Show directory/ies with user data.
  --show-dataserver Show URL of the remote GMT data server.
  --show-doi        Show the DOI for the current release.
  --show-modules    Show all module names.
  --show-library    Show path of the shared GMT library.
  --show-plugindir  Show directory for plug-ins.
  --show-sharedir   Show directory for shared GMT resources.
  --version         Print GMT version number.

if <module-options> is '=' we call exit (0) if module exist and non-zero otherwise.
seisman commented 5 years ago

macOS ships its own libcurl. Maybe GMT.jl found the system libcurl first.

$ otool -L /usr/lib/libcurl.4.dylib
/usr/lib/libcurl.4.dylib:
    /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 9.0.0)
    /usr/lib/libcrypto.42.dylib (compatibility version 43.0.0, current version 43.0.0)
    /usr/lib/libssl.44.dylib (compatibility version 45.0.0, current version 45.1.0)
    /System/Library/Frameworks/LDAP.framework/Versions/A/LDAP (compatibility version 1.0.0, current version 2.4.0)
    /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos (compatibility version 5.0.0, current version 6.0.0)
    /usr/lib/libapple_nghttp2.dylib (compatibility version 1.0.0, current version 1.24.1)
    /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

$ otool -L /Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libcurl.4.dylib
/Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libcurl.4.dylib:
    @executable_path/../lib/libcurl.4.dylib (compatibility version 10.0.0, current version 10.0.0)
    @executable_path/../lib/libidn2.0.dylib (compatibility version 4.0.0, current version 4.6.0)
    @executable_path/../lib/libpsl.5.dylib (compatibility version 9.0.0, current version 9.2.0)
    @executable_path/../lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    @executable_path/../lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0)
    @executable_path/../lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
seisman commented 5 years ago

I have a slightly different error with the same commands:

julia> using Libdl
julia> push!(Libdl.DL_LOAD_PATH, "/Applications/GMT-6.0.0rc4.app/Contents/Resources/lib")
julia> using GMT
julia> coast(region=:global, proj=:Winkel, frame=:g, area=10000,
                    land=:burlywood4, water=:wheat1, figsize=12, show=true)
ERROR: error compiling finish_PS_module: error compiling gmt: could not load library "libgmt"
dlopen(/Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libgmt.dylib, 1): Library not loaded: @executable_path/../lib/libgdal.26.dylib
  Referenced from: /Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libgmt.dylib
  Reason: image not found
joa-quim commented 5 years ago

That’s kind of what I thought. But why isn’t the libgmt.dylib pointing to the GMT’s libcurl? That’s the idea of having a bundle.

Can you please run otool on the libgmt.dylib?

Sent from my iDedo

No dia 17/09/2019, às 14:35, Dongdong Tian notifications@github.com escreveu:

macOS ships its own libcurl. Maybe GMT.jl found the system libcurl first.

$ otool -L /usr/lib/libcurl.4.dylib /usr/lib/libcurl.4.dylib: /usr/lib/libcurl.4.dylib (compatibility version 7.0.0, current version 9.0.0) /usr/lib/libcrypto.42.dylib (compatibility version 43.0.0, current version 43.0.0) /usr/lib/libssl.44.dylib (compatibility version 45.0.0, current version 45.1.0) /System/Library/Frameworks/LDAP.framework/Versions/A/LDAP (compatibility version 1.0.0, current version 2.4.0) /System/Library/Frameworks/Kerberos.framework/Versions/A/Kerberos (compatibility version 5.0.0, current version 6.0.0) /usr/lib/libapple_nghttp2.dylib (compatibility version 1.0.0, current version 1.24.1) /usr/lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)

$ otool -L /Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libcurl.4.dylib /Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libcurl.4.dylib: @executable_path/../lib/libcurl.4.dylib (compatibility version 10.0.0, current version 10.0.0) @executable_path/../lib/libidn2.0.dylib (compatibility version 4.0.0, current version 4.6.0) @executable_path/../lib/libpsl.5.dylib (compatibility version 9.0.0, current version 9.2.0) @executable_path/../lib/libssl.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0) @executable_path/../lib/libcrypto.1.0.0.dylib (compatibility version 1.0.0, current version 1.0.0) @executable_path/../lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11) /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1) — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

seisman commented 5 years ago 
$ otool -L libgmt.dylib
libgmt.dylib:
    @executable_path/../lib/libgmt.6.0.0.dylib (compatibility version 6.0.0, current version 6.0.0)
    @executable_path/../lib/libnetcdf.15.dylib (compatibility version 15.0.0, current version 15.0.0)
    @executable_path/../lib/libcurl.4.dylib (compatibility version 10.0.0, current version 10.0.0)
    @executable_path/../lib/libgdal.26.dylib (compatibility version 27.0.0, current version 27.1.0)
    @executable_path/../lib/libpcre.1.dylib (compatibility version 4.0.0, current version 4.11.0)
    @executable_path/../lib/libfftw3f.3.dylib (compatibility version 9.0.0, current version 9.8.0)
    @executable_path/../lib/libfftw3f_threads.3.dylib (compatibility version 9.0.0, current version 9.8.0)
    @executable_path/../lib/libopenblas-r1.dylib (compatibility version 0.0.0, current version 0.0.0)
    /System/Library/Frameworks/Accelerate.framework/Versions/A/Accelerate (compatibility version 1.0.0, current version 4.0.0)
    @executable_path/../lib/libglib-2.0.0.dylib (compatibility version 5801.0.0, current version 5801.3.0)
    @executable_path/../lib/libpostscriptlight.6.dylib (compatibility version 6.0.0, current version 6.0.0)
    @executable_path/../lib/libz.1.dylib (compatibility version 1.0.0, current version 1.2.11)
    /usr/lib/libSystem.B.dylib (compatibility version 1.0.0, current version 1252.250.1)
joa-quim commented 5 years ago

And let me guess. If you don’t do the step

push!(Libdl.DL_LOAD_PATH (that I needed once but probably should not be needed) and use your own GMT build, then the Julia wrapper works fine.

Sent from my iDedo

No dia 17/09/2019, às 14:45, Dongdong Tian notifications@github.com escreveu:

I have a slightly different error with the same commands:

julia> using Libdl julia> push!(Libdl.DL_LOAD_PATH, "/Applications/GMT-6.0.0rc4.app/Contents/Resources/lib") julia> using GMT julia> coast(region=:global, proj=:Winkel, frame=:g, area=10000, land=:burlywood4, water=:wheat1, figsize=12, show=true) ERROR: error compiling finish_PS_module: error compiling gmt: could not load library "libgmt" dlopen(/Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libgmt.dylib, 1): Library not loaded: @executable_path/../lib/libgdal.26.dylib Referenced from: /Applications/GMT-6.0.0rc4.app/Contents/Resources/lib/libgmt.dylib Reason: image not found — You are receiving this because you commented. Reply to this email directly, view it on GitHub, or mute the thread.

seisman commented 5 years ago

Yes, the julia wrapper works with my own GMT build.

joa-quim commented 5 years ago

Thanks. So the question is what is OSX doing? It has a lib with rpath pointing to other libs but depending on how that lib is accessed the rpath are respected ... or not!!!!

seisman commented 5 years ago

PyGMT also has the some problem with the DMG package, but works with my own build.

joa-quim commented 5 years ago

If PyGMT also accesses the gmt lib via an FFI then the problem should be the same.

PaulWessel commented 5 years ago

This is similar to the issue we have with Matlab where Matlab tells gmt to use its netcdf despite libgmt pointing to another libnetcdf.

joa-quim commented 5 years ago

Yes, except that now we can't blame TMW.

PaulWessel commented 5 years ago

We may need help from someone who understands how shared libraries find other shared libraries and how that is affected by linker flags. There has to be a way to prevent some rogue application from overriding the sequence of dependency in our shared library.

seisman commented 5 years ago

FYI, GMT from homebrew works for both GMT.jl and PyGMT.

PaulWessel commented 5 years ago

The man ld command on macOS talks about linking and how symbols are resolved. I wonder if some of those options are needed but cannot fully understand what to do.

PaulWessel commented 5 years ago

If we could figure out how this works we may solve the long-standing crap with MATLAB as well. We should try to understand link options better.

joa-quim commented 5 years ago

@ZMAlt As you see OSX plays mysterious tricks with us. I'm afraid that for the moment the recommendation install GMT from any of the other alternatives but the bundle. Because 6.0.0 is not yet officially released Homebrew does not have the formula yet, but, if you have homebrew, you can install GMTrc4 with

brew install https://raw.githubusercontent.com/claudiodsf/homebrew-core/GMT6.0.0/Formula/gmt.rb
joa-quim commented 5 years ago

One thing that surprised me in the otool output was that it says @executable_path/.. but shouldn't it be @rpath ...? I thought (as if could understand anything that is described by the dylib man page (example)) rpath was the way to create relocatables.

This is issue is also slightly different from the MEX one. Here the normal build works and it's the bundle, who also tries to protect from the dll hell, that doesn't.

ZMAlt commented 5 years ago

@joa-quim OK! Thanks

giordano commented 5 years ago

@joa-quim have you considered using BinaryBuilder to provide the library?

joa-quim commented 5 years ago

@giordano Although GMT can be built without GDAL it would be too crippled without it and GDAL is a complex dependency (for example GDAL.jl is rather limited in drivers). On the other hand BinaryBuilder would require a MinGW build on Windows, which is something I'm not willing to accept. I understand how that would easy life for many users but for the moment out of my reach.

Let's see how the new Artifacts in Julia 1.3 can change this game. But for now just install with homebrew as indicated above. It's soooooo easy.