Doesn't find libz on FreeBSD for CodecZlib.jl

[yurivict]

CodecZlib.jl requires libz: https://github.com/bicycle1885/CodecZlib.jl/blob/master/deps/build.jl

However, on FreeBSD it still downloads sources and breaks, even though /usr/lib/libz.so exists.

[ararslan]

Can you post the output of versioninfo(true)?

[yurivict]

julia> versioninfo(true)
Julia Version 0.6.2
Commit d386e40 (2017-12-13 18:08 UTC)
Platform Info:
  OS: FreeBSD (amd64-unknown-freebsd11.1)
  CPU: Intel(R) Core(TM) i7 CPU         930  @ 2.80GHz
  WORD_SIZE: 64
  uname: FreeBSD 11.1-STABLE FreeBSD 11.1-STABLE #0 r331609M: Tue Mar 27 03:33:04 PDT 2018     unknown@nohost.com:/oldie/obj/usr/src/sys/GENERIC  amd64 amd64
Memory: 23.934940338134766 GB (321.91796875 MB free)
Uptime: 320216.0 sec
Load Avg:  0.4375  0.45654296875  0.4580078125
Intel(R) Core(TM) i7 CPU         930  @ 2.80GHz: 
       speed         user         nice          sys         idle          irq
#1  2806 MHz     204050 s        251 s      24438 s    1984612 s      10649 s
#2  2806 MHz     254012 s        167 s      29348 s    1939954 s        518 s
#3  2806 MHz     258583 s        201 s      31387 s    1933118 s        710 s
#4  2806 MHz     257770 s        190 s      30991 s    1934357 s        691 s
#5  2806 MHz     229372 s        186 s      27539 s    1958375 s       8527 s
#6  2806 MHz     257351 s        193 s      30855 s    1934937 s        663 s
#7  2806 MHz     234426 s        188 s      26923 s    1958605 s       3859 s
#8  2806 MHz     258157 s        203 s      30379 s    1934538 s        721 s

  BLAS: libopenblas (DYNAMIC_ARCH NO_AFFINITY Nehalem)
  LAPACK: libopenblasp
  LIBM: libm
  LLVM: libLLVM-4.0.1 (ORCJIT, nehalem)
Environment:
  GLADE_CATALOG_PATH = :
  GLADE_MODULE_PATH = :
  HOME = /home/yuri
  GLADE_PIXMAP_PATH = :
  TERM = xterm-256color
  PATH = .:/home/yuri/bin:/sbin:/bin:/usr/sbin:/usr/bin:/usr/local/sbin:/usr/local/bin

Package Directory: /home/yuri/.julia/v0.6
7 required packages:
 - ASCIIPlots                    0.0.3
 - AbstractNumbers               0.1.1
 - Bokeh                         0.2.0
 - CSV                           0.2.3
 - Crayons                       0.4.1
 - Crypto                        0.0.1              pinned.82c8a3b0.tmp
 - ZMQ                           0.5.1
25 additional packages:
 - BinDeps                       0.8.7
 - BinaryProvider                0.2.7
 - CategoricalArrays             0.3.6
 - CodecZlib                     0.4.3
 - Compat                        0.63.0
 - Coverage                      0.5.3
 - DataFrames                    0.11.5
 - DataStreams                   0.3.4
 - DataStructures                0.7.4
 - Dates                         0.4.4
 - HTTP                          0.6.8
 - IniFile                       0.4.0
 - JSON                          0.8.3
 - MbedTLS                       0.5.8
 - Missings                      0.2.8
 - Mustache                      0.3.1
 - NamedTuples                   4.0.0
 - Reexport                      0.1.0
 - SHA                           0.5.6
 - SortingAlgorithms             0.2.0
 - SpecialFunctions              0.3.8
 - StatsBase                     0.20.1
 - TranscodingStreams            0.5.2
 - URIParser                     0.3.1
 - WeakRefStrings                0.4.3

[ararslan]

Thanks. Can you post the full error message and stack trace from Pkg.build("CodecZlib")? I implemented the source build fallback for CodecZlib and it worked for me on FreeBSD 11.1 with Julia master.

[yurivict]

julia> Pkg.add("CSV")
INFO: Cloning cache of BinDeps from https://github.com/JuliaLang/BinDeps.jl.git
INFO: Cloning cache of BinaryProvider from https://github.com/JuliaPackaging/BinaryProvider.jl.git
INFO: Cloning cache of CSV from https://github.com/JuliaData/CSV.jl.git
INFO: Cloning cache of CategoricalArrays from https://github.com/JuliaData/CategoricalArrays.jl.git
INFO: Cloning cache of CodecZlib from https://github.com/bicycle1885/CodecZlib.jl.git
INFO: Cloning cache of DataFrames from https://github.com/JuliaData/DataFrames.jl.git
INFO: Cloning cache of DataStreams from https://github.com/JuliaData/DataStreams.jl.git
INFO: Cloning cache of DataStructures from https://github.com/JuliaCollections/DataStructures.jl.git
INFO: Cloning cache of Missings from https://github.com/JuliaData/Missings.jl.git
INFO: Cloning cache of NamedTuples from https://github.com/blackrock/NamedTuples.jl.git
INFO: Cloning cache of Reexport from https://github.com/simonster/Reexport.jl.git
INFO: Cloning cache of SHA from https://github.com/staticfloat/SHA.jl.git
INFO: Cloning cache of SortingAlgorithms from https://github.com/JuliaCollections/SortingAlgorithms.jl.git
INFO: Cloning cache of SpecialFunctions from https://github.com/JuliaMath/SpecialFunctions.jl.git
INFO: Cloning cache of StatsBase from https://github.com/JuliaStats/StatsBase.jl.git
INFO: Cloning cache of TranscodingStreams from https://github.com/bicycle1885/TranscodingStreams.jl.git
INFO: Cloning cache of URIParser from https://github.com/JuliaWeb/URIParser.jl.git
INFO: Cloning cache of WeakRefStrings from https://github.com/JuliaData/WeakRefStrings.jl.git
INFO: Installing BinDeps v0.8.7
INFO: Installing BinaryProvider v0.2.7
INFO: Installing CSV v0.2.3
INFO: Installing CategoricalArrays v0.3.6
INFO: Installing CodecZlib v0.4.3
INFO: Installing DataFrames v0.11.5
INFO: Installing DataStreams v0.3.4
INFO: Installing DataStructures v0.7.4
INFO: Installing Missings v0.2.8
INFO: Installing NamedTuples v4.0.0
INFO: Installing Reexport v0.1.0
INFO: Installing SHA v0.5.6
INFO: Installing SortingAlgorithms v0.2.0
INFO: Installing SpecialFunctions v0.3.8
INFO: Installing StatsBase v0.20.1
INFO: Installing TranscodingStreams v0.5.2
INFO: Installing URIParser v0.3.1
INFO: Installing WeakRefStrings v0.4.3
INFO: Building CodecZlib
WARNING: Platform `amd64-unknown-freebsd11.1` is not an officially supported platform
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  593k  100  593k    0     0   798k      0 --:--:-- --:--:-- --:--:--  797k
Checking for gcc...
Checking for shared library support...
No shared library support.
Building static library libz.a version 1.2.11 with cc.
Checking for size_t... Yes.
Checking for off64_t... Yes.
Checking for fseeko... Yes.
Checking for strerror... Yes.
Checking for unistd.h... Yes.
Checking for stdarg.h... Yes.
Checking whether to use vs[n]printf() or s[n]printf()... using vs[n]printf().
Checking for vsnprintf() in stdio.h... Yes.
Checking for return value of vsnprintf()... Yes.
cc -O -D_LARGEFILE64_SOURCE=1 -I. -c -o example.o test/example.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o adler32.o adler32.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o crc32.o crc32.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o deflate.o deflate.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o infback.o infback.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o inffast.o inffast.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o inflate.o inflate.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o inftrees.o inftrees.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o trees.o trees.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o zutil.o zutil.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o compress.o compress.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o uncompr.o uncompr.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o gzclose.o gzclose.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o gzlib.o gzlib.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o gzread.o gzread.c
cc -O -D_LARGEFILE64_SOURCE=1  -c -o gzwrite.o gzwrite.c
cc -O -D_LARGEFILE64_SOURCE=1 -I. -c -o minigzip.o test/minigzip.c
cc -O -D_LARGEFILE64_SOURCE=1 -I. -D_FILE_OFFSET_BITS=64 -c -o example64.o test/example.c
cc -O -D_LARGEFILE64_SOURCE=1 -I. -D_FILE_OFFSET_BITS=64 -c -o minigzip64.o test/minigzip.c
ar rc libz.a adler32.o crc32.o deflate.o infback.o inffast.o inflate.o inftrees.o trees.o zutil.o compress.o uncompr.o gzclose.o gzlib.o gzread.o gzwrite.o 
cc -O -D_LARGEFILE64_SOURCE=1 -o example example.o -L. libz.a
cc -O -D_LARGEFILE64_SOURCE=1 -o minigzip minigzip.o -L. libz.a
cc -O -D_LARGEFILE64_SOURCE=1 -o example64 example64.o -L. libz.a
cc -O -D_LARGEFILE64_SOURCE=1 -o minigzip64 minigzip64.o -L. libz.a
============================================================================[ ERROR: CodecZlib ]=============================================================================

LoadError: zlib was unable to build properly
while loading /usr/home/yuri/.julia/v0.6/CodecZlib/deps/build.jl, in expression starting on line 86

=============================================================================================================================================================================
INFO: Building SpecialFunctions

==============================================================================[ BUILD ERRORS ]===============================================================================

WARNING: CodecZlib had build errors.

 - packages with build errors remain installed in /usr/home/yuri/.julia/v0.6
 - build the package(s) and all dependencies with `Pkg.build("CodecZlib")`
 - build a single package by running its `deps/build.jl` script

=============================================================================================================================================================================
INFO: Package database updated

[ararslan]

No shared library support. Building static library libz.a version 1.2.11 with cc.

It's failing because it can't find the built shared library, which is not surprising given that it apparently can't build shared libraries at all...

BinaryProvider doesn't support FreeBSD (yet! see https://github.com/JuliaPackaging/BinaryBuilder.jl/issues/32) and the issue is with the source fallback code, which is specific to CodecZlib, so this issue should be closed here and reopened as an issue on CodecZlib.

[yurivict]

Julia can't be used on FreeBSD much then, because random packages break.

[ararslan]

To be fair, this seems like a configuration issue; based on the output I think zlib wants GCC specifically, which is not installed by default on FreeBSD. I recommend installing GCC from ports then trying again. You'll need GCC anyway for Julia packages with Fortran dependencies, such as SpecialFunctions.

There hasn't been a big push for FreeBSD support in Julia until fairly recently, and there's still some room for improvement. (When I say "big push," I really mean that a couple of people have put a lot of effort into it.) It's always good to have more people on FreeBSD using Julia so that weak points can be identified and improved.

[yurivict]

I have gcc installed: gcc6 executable. libz.so though was built with clang, and nothing around it should require gcc.

FreeBSD is different mostly in that random packages don't build well from sources. This is because they are mostly geared to work on linux.

[ararslan]

What happens if you follow the steps as they're written in the source build fallback? Basically

curl -sLO https://zlib.net/zlib-1.2.11.tar.gz
tar xzf zlib-1.2.11.tar.gz
cd zlib-1.2.11
./configure --prefix=.
gmake

Could be that the source build is improperly specified or otherwise insufficiently general.

[yurivict]

These steps succeed.

[ararslan]

Does it produce libz.so?

[yurivict]

It produces libz.a when called like this: CC=cc CFLAGS=-O2 ./configure --enable-shared --prefix=. It needs --enable-shared and CC and CFLAGS for some reason.

[ararslan]

Ah, --enable-shared is important. I'm actually surprised it worked for me without that when I wrote it. The fact that it needs CC and CFLAGS is a little weird though...

[yurivict]

It is weird that it depends on CC/CFLAGS. But it's best if CC and CFLAGS are always set. Otherwise, how would it ever know which compiler and optimization to use...