Closed logankilpatrick closed 1 year ago
I don't think this is fixed.
The Apple-provided libz for whatever reason does not provide gzopen64
, only gzopen
:
> grep gzopen /Library/Developer/CommandLineTools/SDKs/MacOSX13.3.sdk/usr/lib/libz.1.tbd
_gzgetc, _gzgetc_, _gzgets, _gzoffset, _gzopen, _gzprintf,
In https://github.com/JuliaIO/GZip.jl/blob/57cff17fe791e6befdb544922fecc0601144c513/src/gz.jl#L158C34-L158C34 there is a check if the gzopen64
symbol can be found, otherwise gzopen
is used.
But it's no use, because here https://github.com/JuliaIO/GZip.jl/blob/57cff17fe791e6befdb544922fecc0601144c513/src/lib/aarch64-apple-darwin20.jl#L481C24-L481C24 gzopen
is set to use gzopen64
once again.
The result is that GZip fails on macOS:
ERROR: could not load symbol "gzopen64":
dlsym(0x3a6260d58, gzopen64): symbol not found
Stacktrace:
[1] gzopen64
@ ~/.julia/packages/GZip/I6uhZ/src/lib/aarch64-apple-darwin20.jl:92 [inlined]
[2] gzopen(fname::String, gzmode::String, gz_buf_size::Int64)
@ GZip ~/.julia/packages/GZip/I6uhZ/src/gz.jl:210
[3] gzopen
@ ~/.julia/packages/GZip/I6uhZ/src/gz.jl:226 [inlined]
[4] open
@ ~/.julia/packages/GZip/I6uhZ/src/gz.jl:233 [inlined]
Workaround is installing zlib
from homebrew and starting julia with DYLD_LIBRARY_PATH=/opt/homebrew/opt/zlib/lib
.
Julia should really be using its own gzip and not the apple provided one. Are you using the latest GZip.jl? I don't have this problem and I have the same architecture. Do you have some other env variables that are interfering perhaps?
Now that we are building arch-specific binaries, we should not need the first check - and get rid of the _gzopen
etc. But if we do that, it will always use gzopen64
on mac arm64.
Just for fun, I have a branch to try out zlib-ng, which may be a better solution.
@linearray Can you try the latest master? Also, we explicitly use Zlib_jll.libz_path
in every ccall
statement. What is pulling in the system zlib here? Can you check you have no other LD_LIBRARY_PATH
or DYLD_LIBRARY_PATH
?
This works fine on my arm64 mac.
I run GZip.jl 0.6.0, installed julia 1.9.2 from homebrew and don't use any relevant envvars.
julia> import Zlib_jll
julia> Zlib_jll.libz_path
"/usr/lib/libz.1.dylib"
Apparently the problem sits deeper. What it the value of Zlib_jll.libz_path
on your system?
This is what it should be for everyone using official binaries/build process:
julia> Zlib_jll.libz_path
"/Users/viral/.julia/juliaup/julia-1.9.2+0.aarch64.apple.darwin14/lib/julia/libz.1.2.13.dylib"
Are you using Julia binaries that are provided by homebrew (not cask) which links to system zlib?
Correct, I just did brew install julia
.
So is this in fact a homebrew bug?
Yes, homebrew likes to build Julia differently than the official distribution, reusing libraries built by homebrew and system libraries. I would recommend using juliaup
or install the official binaries directly.
We could possibly add a check for this case and perhaps issue a warning. The other possibility is to use zlib-ng
(#104), which I am very close to and will avoid this altogether.
Per here: https://github.com/JuliaML/MLDatasets.jl/issues/86 it looks like this is failing on macOS Monterey. If I run
test GZip
in Julia 1.6 I get the same error as I got in the issue linked above: