search

deephealthproject / eddl

European Distributed Deep Learning (EDDL) library. A general-purpose library initially developed to cover deep learning needs in healthcare use cases within the DeepHealth project.
https://deephealthproject.github.io/eddl/
MIT License
34 stars 10 forks source link

Superbuild failure if zlib is not manually installed #149

Closed simleo closed 4 years ago

simleo commented 4 years ago

If my understanding is correct, the new superbuild mode should automatically get all dependencies needed to build the library. However, it looks like zlib still needs to be installed beforehand.

Starting from scratch on Ubuntu 18.04 I only install the following: build-essential, ca-certificates, wget, git and cmake (3.14.5). I run cmake with cmake -D BUILD_SUPERBUILD=ON -DBUILD_TESTS=OFF, and the output shows zlib being fetched and installed (in a cmake/third_party/zlib relative dir). However, make eventually fails with:

make[2]: Entering directory '/eddl/build'
[ 77%] Building CXX object examples/CMakeFiles/onnx_export.dir/onnx/2_onnx_save_net_to_file.cpp.o
cd /eddl/build/examples && /usr/bin/c++  -DOpenMP_VERSION_MAJOR=4 -I/eddl/include -isystem /eddl/build/cmake/third_party/eigen/include/eigen3  -O3 -fPIE   -fopenmp -std=gnu++11 -o CMakeFiles/onnx_export.dir/onnx/2_onnx_save_net_to_file.cpp.o -c /eddl/examples/onnx/2_onnx_save_net_to_file.cpp
In file included from /eddl/include/eddl/tensor/tensor.h:26:0,
                 from /eddl/include/eddl/initializers/initializer.h:16,
                 from /eddl/include/eddl/layers/layer.h:17,
                 from /eddl/include/eddl/net/net.h:17,
                 from /eddl/include/eddl/apis/eddl.h:19,
                 from /eddl/examples/onnx/2_onnx_save_net_to_file.cpp:14:
/eddl/include/eddl/tensor/cnpy/cnpy.h:16:10: fatal error: zlib.h: No such file or directory
 #include <zlib.h>
          ^~~~~~~~
compilation terminated.
examples/CMakeFiles/onnx_export.dir/build.make:65: recipe for target 'examples/CMakeFiles/onnx_export.dir/onnx/2_onnx_save_net_to_file.cpp.o' failed
make[2]: *** [examples/CMakeFiles/onnx_export.dir/onnx/2_onnx_save_net_to_file.cpp.o] Error 1
make[2]: Leaving directory '/eddl/build'
CMakeFiles/Makefile2:212: recipe for target 'examples/CMakeFiles/onnx_export.dir/all' failed
make[1]: *** [examples/CMakeFiles/onnx_export.dir/all] Error 2
make[1]: Leaving directory '/eddl/build'
Makefile:132: recipe for target 'all' failed
make: *** [all] Error 2

If I also install zlib1g-dev before running cmake and make, instead, there are no errors.

All of the above has been tried with EDDL v0.5.4a.

salvacarrion commented 4 years ago

Show me the summary from cmake

simleo commented 4 years ago

Adding the cmake output as an attachment. eddl_superbuild_nozlib_cmake.log

salvacarrion commented 4 years ago

It looks correct. Can you send the output of the compilation?

-- ===========================================
-- ===========================================
-- Project name: eddl
-- Project version: 0.5
-- Prefix path: 
-- Install prefix: /usr/local
-- -------------------------------------------
-- Build target: CPU
-- Build type: Release
-- Build shared libs: ON
-- Build coverage: OFF
-- -------------------------------------------
-- C++ compiler: GNU (/usr/bin/c++)
-- C++ flags: 
-- C++ flags (release): -O3
-- C++ flags (debug): -O0 -g
-- -------------------------------------------
-- CUDA enabled: OFF
-- -------------------------------------------
-- OpenMP enabled: ON
-- OpenMP version: 4
-- OpenMP gomp library: /usr/lib/gcc/x86_64-linux-gnu/7/libgomp.so
-- OpenMP pthread library: /usr/lib/x86_64-linux-gnu/libpthread.so
-- -------------------------------------------
-- Eigen3 dir: /eddl/build/cmake/third_party/eigen/share/eigen3/cmake
-- Eigen3 include: /eddl/build/cmake/third_party/eigen/include/eigen3
-- -------------------------------------------
-- ZLIB root: /eddl/build/cmake/third_party/zlib
-- ZLIB include: /eddl/build/cmake/third_party/zlib/include
-- ZLIB libraries: z
-- -------------------------------------------
-- Use Protobuf: ON
-- Protobuf dir: /eddl/build/cmake/third_party/protobuf/lib/cmake/protobuf
-- Protobuf include: /eddl/build/cmake/third_party/protobuf/include
-- Protobuf libraries: protobuf
-- Protobuf compiler: /eddl/build/cmake/third_party/protobuf/bin/protoc
-- ===========================================
-- ===========================================
-- The C compiler identification is GNU 7.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Configuring done
-- Generating done
-- Build files have been written to: /eddl/build
simleo commented 4 years ago

make output attached. eddl_superbuild_nozlib_make.log

CostantinoGrana commented 4 years ago

@simleo Can you check this again, after the changes made in the previous pull request (already merged in master).

simleo commented 4 years ago

@CostantinoGrana I checked again on the master branch. Now the installation works without manually installing zlib1g-dev. However:

# ldd /usr/local/lib/libeddl.so 
    linux-vdso.so.1 (0x00007ffee4df5000)
    libz.so.1 => /lib/x86_64-linux-gnu/libz.so.1 (0x00007fe33f4f7000)
    libprotobuf.so.3.11.4.0 => not found
    libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007fe33f2d8000)
    libgomp.so.1 => /usr/lib/x86_64-linux-gnu/libgomp.so.1 (0x00007fe33f0a9000)
    libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007fe33ed20000)
    libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007fe33e982000)
    libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007fe33e76a000)
    libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007fe33e379000)
    /lib64/ld-linux-x86-64.so.2 (0x00007fe33fb47000)
    libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007fe33e175000)

libz and libprotobuf are not statically linked anymore, which I think was @salvacarrion 's purpose with the superbuild.

salvacarrion commented 4 years ago

Yes, that was the main purpose of the superbuild since the conda environment already worked pretty well.

You can ignore the zlib static linking because it is only used to read Numpy arrays (for protobuf is disabled) and pretty soon we will remove the numpy support (on the C++ side) as it is not officially supported.

CostantinoGrana commented 4 years ago

@simleo Wait, isn't the purpose of BUILD_SHARED_LIBS to tell that you want shared libs? If you set it (default) zlib should use the shared version, right?

salvacarrion commented 4 years ago

Sort of, but with a nuance for those dependencies. The point of setting the other libraries as static with the fPIC flag (only for the superbuild), was so that they could be embedded in the EDDL library. Then, this EDDL library could be either shared or static.

salvacarrion commented 4 years ago

I'm closing this issue to open a more general one