Failing to precompile Snappy.jl.....

[calebwin]

I'm trying to add the latest version of Parquet.jl to my project but I keep getting the following error:

julia> using Parquet
[ Info: Precompiling Parquet [626c502c-15b0-58ad-a749-f091afb673ae]
ERROR: LoadError: Snappy not installed properly, run Pkg.build("Snappy"), restart Julia and try again
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] top-level scope
   @ ~/.julia/packages/Snappy/O2CHr/src/Snappy.jl:8
 [3] include
   @ ./Base.jl:386 [inlined]
 [4] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::String)
   @ Base ./loading.jl:1213
 [5] top-level scope
   @ none:1
 [6] eval
   @ ./boot.jl:360 [inlined]
 [7] eval(x::Expr)
   @ Base.MainInclude ./client.jl:446
 [8] top-level scope
   @ none:1
in expression starting at /root/.julia/packages/Snappy/O2CHr/src/Snappy.jl:3
ERROR: LoadError: Failed to precompile Snappy [59d4ed8c-697a-5b28-a4c7-fe95c22820f9] to /root/.julia/compiled/v1.6/Snappy/jl_LERQIG.
Stacktrace:
  [1] error(s::String)
    @ Base ./error.jl:33
  [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::Base.TTY, internal_stdout::Base.TTY)
    @ Base ./loading.jl:1360
  [3] compilecache(pkg::Base.PkgId, path::String)
    @ Base ./loading.jl:1306
  [4] _require(pkg::Base.PkgId)
    @ Base ./loading.jl:1021
  [5] require(uuidkey::Base.PkgId)
    @ Base ./loading.jl:914
  [6] require(into::Module, mod::Symbol)
    @ Base ./loading.jl:901
  [7] include
    @ ./Base.jl:386 [inlined]
  [8] include_package_for_output(pkg::Base.PkgId, input::String, depot_path::Vector{String}, dl_load_path::Vector{String}, load_path::Vector{String}, concrete_deps::Vector{Pair{Base.PkgId, UInt64}}, source::Nothing)
    @ Base ./loading.jl:1213
  [9] top-level scope
    @ none:1
 [10] eval
    @ ./boot.jl:360 [inlined]
 [11] eval(x::Expr)
    @ Base.MainInclude ./client.jl:446
 [12] top-level scope
    @ none:1
in expression starting at /root/.julia/packages/Parquet/O0PXc/src/Parquet.jl:1
ERROR: Failed to precompile Parquet [626c502c-15b0-58ad-a749-f091afb673ae] to /root/.julia/compiled/v1.6/Parquet/jl_gdBXYO.
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] compilecache(pkg::Base.PkgId, path::String, internal_stderr::Base.TTY, internal_stdout::Base.TTY)
   @ Base ./loading.jl:1360
 [3] compilecache(pkg::Base.PkgId, path::String)
   @ Base ./loading.jl:1306
 [4] _require(pkg::Base.PkgId)
   @ Base ./loading.jl:1021
 [5] require(uuidkey::Base.PkgId)
   @ Base ./loading.jl:914
 [6] require(into::Module, mod::Symbol)
   @ Base ./loading.jl:901

The issue seems to be with Snappy.jl:

julia> using Pkg; Pkg.build("Snappy")
    Building Snappy → `~/.julia/scratchspaces/44cfe95a-1eb2-52ea-b672-e2afdf69b78f/25620a91907972a05863941d6028791c2613888e/build.log`
ERROR: Error building `Snappy`: 
[ Info: Directory /root/.julia/packages/Snappy/O2CHr/deps/usr/lib does not exist!
┌ Warning: Could not extract the platform key of https://github.com/davidanthoff/SnappyBuilder/releases/download/v1.1.7%2Bbuild.1/SnappyBuilder.x86_64-linux-gnu.tar.gz; continuing...
└ @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/Prefix.jl:185
[ Info: Downloading https://github.com/davidanthoff/SnappyBuilder/releases/download/v1.1.7%2Bbuild.1/SnappyBuilder.x86_64-linux-gnu.tar.gz to /root/.julia/packages/Snappy/O2CHr/deps/usr/downloads/SnappyBuilder.x86_64-linux-gnu.tar.gz...
ERROR: LoadError: Could not download https://github.com/davidanthoff/SnappyBuilder/releases/download/v1.1.7%2Bbuild.1/SnappyBuilder.x86_64-linux-gnu.tar.gz to /root/.julia/packages/Snappy/O2CHr/deps/usr/downloads/SnappyBuilder.x86_64-linux-gnu.tar.gz:
ErrorException("")
Stacktrace:
 [1] error(s::String)
   @ Base ./error.jl:33
 [2] download(url::String, dest::String; verbose::Bool)
   @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformEngines.jl:619
 [3] download_verify(url::String, hash::String, dest::String; verbose::Bool, force::Bool, quiet_download::Bool)
   @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformEngines.jl:688
 [4] install(tarball_url::String, hash::String; prefix::Prefix, tarball_path::String, force::Bool, ignore_platform::Bool, verbose::Bool)
   @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/Prefix.jl:314
 [5] top-level scope
   @ ~/.julia/packages/Snappy/O2CHr/deps/build.jl:28
 [6] include(fname::String)
   @ Base.MainInclude ./client.jl:444
 [7] top-level scope
   @ none:5
in expression starting at /root/.julia/packages/Snappy/O2CHr/deps/build.jl:24

caused by: 
Stacktrace:
 [1] error()
   @ Base ./error.jl:42
 [2] download(url::String, dest::String; verbose::Bool)
   @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformEngines.jl:613
 [3] download_verify(url::String, hash::String, dest::String; verbose::Bool, force::Bool, quiet_download::Bool)
   @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/PlatformEngines.jl:688
 [4] install(tarball_url::String, hash::String; prefix::Prefix, tarball_path::String, force::Bool, ignore_platform::Bool, verbose::Bool)
   @ BinaryProvider ~/.julia/packages/BinaryProvider/U2dKK/src/Prefix.jl:314
 [5] top-level scope
   @ ~/.julia/packages/Snappy/O2CHr/deps/build.jl:28
 [6] include(fname::String)
   @ Base.MainInclude ./client.jl:444
 [7] top-level scope
   @ none:5
[15:50:31] Warning: Failed to create the file ##################################### 100.0%##O#-#                                                                        
[15:50:31] Warning: /root/.julia/packages/Snappy/O2CHr/deps/usr/downloads/SnappyBuilder.x8
[15:50:31] Warning: 6_64-linux-gnu.tar.gz: Permission denied
[15:50:31] ##                                                                         3.1%curl: (23) Failure writing output to destination
[15:50:31] 
Stacktrace:
  [1] pkgerror(msg::String)
    @ Pkg.Types /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Types.jl:55
  [2] (::Pkg.Operations.var"#82#87"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec})()
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1044
  [3] withenv(::Pkg.Operations.var"#82#87"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, ::Pair{String, String}, ::Vararg{Pair{String, B} where B, N} where N)
    @ Base ./env.jl:161
  [4] (::Pkg.Operations.var"#109#113"{String, Pkg.Operations.var"#82#87"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, Pkg.Types.PackageSpec})()
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1542
  [5] with_temp_env(fn::Pkg.Operations.var"#109#113"{String, Pkg.Operations.var"#82#87"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, Pkg.Types.PackageSpec}, temp_env::String)
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1444
  [6] (::Pkg.Operations.var"#108#112"{Pkg.Operations.var"#82#87"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, Pkg.Types.Context, Pkg.Types.PackageSpec, String, Pkg.Types.Project, String})(tmp::String)
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1517
  [7] mktempdir(fn::Pkg.Operations.var"#108#112"{Pkg.Operations.var"#82#87"{Bool, Pkg.Types.Context, String, Pkg.Types.PackageSpec}, Pkg.Types.Context, Pkg.Types.PackageSpec, String, Pkg.Types.Project, String}, parent::String; prefix::String)
    @ Base.Filesystem ./file.jl:729
  [8] mktempdir(fn::Function, parent::String) (repeats 2 times)
    @ Base.Filesystem ./file.jl:727
  [9] sandbox(fn::Function, ctx::Pkg.Types.Context, target::Pkg.Types.PackageSpec, target_path::String, sandbox_path::String, sandbox_project_override::Pkg.Types.Project)
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1483
 [10] build_versions(ctx::Pkg.Types.Context, uuids::Vector{Base.UUID}; verbose::Bool)
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:1025
 [11] build(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}, verbose::Bool)
    @ Pkg.Operations /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/Operations.jl:906
 [12] build(ctx::Pkg.Types.Context, pkgs::Vector{Pkg.Types.PackageSpec}; verbose::Bool, kwargs::Base.Iterators.Pairs{Symbol, Base.TTY, Tuple{Symbol}, NamedTuple{(:io,), Tuple{Base.TTY}}})
    @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:899
 [13] build(pkgs::Vector{Pkg.Types.PackageSpec}; io::Base.TTY, kwargs::Base.Iterators.Pairs{Union{}, Union{}, Tuple{}, NamedTuple{(), Tuple{}}})
    @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:79
 [14] build(pkgs::Vector{Pkg.Types.PackageSpec})
    @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:77
 [15] #build#71
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:75 [inlined]
 [16] build
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:75 [inlined]
 [17] #build#70
    @ /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:74 [inlined]
 [18] build(pkg::String)
    @ Pkg.API /buildworker/worker/package_linux64/build/usr/share/julia/stdlib/v1.6/Pkg/src/API.jl:74
 [19] top-level scope
    @ REPL[8]:1

I've tried removing and re-adding the latest version of Parquet.jl but to no avail.

Any ideas on how to debug this?

[calebwin]

Does Parquet.jl need to use this (unmerged and unreleased) version of Snappy.jl - https://github.com/JuliaLang/julia/issues/40299 ?

[calebwin]

Okay I realized my issue. I had libsnappy installed (using sudo apt-get install -y libsnappy-dev) but in the wrong place. A simple symbolic link put it in the right place - udo ln -s /lib/x86_64-linux-gnu/libsnappy.so.1 /home/calebwin/.julia/packages/Snappy/O2CHr/deps/usr/lib/libsnappy.so.1.

I still don't know why the build script failed. So that might be a thing for someone to look into at some point (I can't atm unfortunately).

[calebwin]

As it turns out, I also needed:

calebwin@calebwin-XPS-13-9310:~$ sudo mkdir /root/.julia/packages/Snappy/O2CHr/deps/usr/lib
calebwin@calebwin-XPS-13-9310:~$ sudo ln -s /usr/lib/x86_64-linux-gnu/libsnappy.so /root/.julia/packages/Snappy/O2CHr/deps/usr/lib/libsnappy.so

Very confusing...

[calebwin]

I'm hoping https://github.com/bicycle1885/Snappy.jl/pull/25 resolves this but I'm not sure it worked when I tried using it...

[calebwin]

Also, note that you probably won't be able to replicate this issue unless you don't have libsnappy-dev installed properly already.