search

grimbough / Rhdf5lib

Distribution of the HDF5 library in an R package
https://bioconductor.org/packages/Rhdf5lib/
6 stars 14 forks source link

Failing to install Rhdf5lib: 'cannot find -lldap' and 'cannot find -llber' #34

Open yentlprojects opened 3 years ago

yentlprojects commented 3 years ago

On a linux server I installed BiocManager to a specific R_LIBS directory and am now trying to install the flowCore library. I am using R version 4.0.2 and Bioconductor version 3.12.

Initally I did this using Rscript -e "BiocManager::install('flowCore')"

This resulted in errors for flowCore not finding cytolib, which in turn couldn't find Rhdf5lib, due to errors with finding -lldap and -llber. Next I tried to install Rhdf5lib specifically using: Rscript -e "BiocManager::install('grimbough/Rhdf5lib')" and Rscript -e "BiocManager::install('grimbough/Rhdf5lib', INSTALL_opts = '--force-biarch')"

which both resulted in the same output (can provide full log if needed):

  CC       H5HLdblk.lo
  CC       H5HP.lo
  CC       H5I.lo
  CC       H5Itest.lo
  CC       H5L.lo
  CC       H5Lexternal.lo
  CC       H5make_libsettings.o
  CCLD     H5make_libsettings
/apps/leuven/skylake/2018a/software/binutils/2.28-GCCcore-6.4.0/bin/ld.gold: error: cannot find -lldap
/apps/leuven/skylake/2018a/software/binutils/2.28-GCCcore-6.4.0/bin/ld.gold: error: cannot find -llber
collect2: error: ld returned 1 exit status
make[2]: *** [H5make_libsettings] Error 1
make[2]: Leaving directory `/tmp/RtmpWh6BJm/R.INSTALL10401ea40216/Rhdf5lib/src/hdf5/src'
make[1]: *** [lib] Error 1
make[1]: Leaving directory `/tmp/RtmpWh6BJm/R.INSTALL10401ea40216/Rhdf5lib/src/hdf5/src'
make: *** [lib] Error 1
configure:    HDF5_INCLUDE=hdf5/src
configure:    HDF5_CXX_INCLUDE=hdf5/c++/src
configure:    HDF5_HL_INCLUDE=hdf5/hl/src
configure:    HDF5_HL_CXX_INCLUDE=hdf5/hl/c++/src
configure:    HDF5_LIB=hdf5/src/.libs/libhdf5.a
configure:    HDF5_CXX_LIB=hdf5/c++/src/.libs/libhdf5_cpp.a
configure:    HDF5_HL_LIB=hdf5/hl/src/.libs/libhdf5_hl.a
configure:    HDF5_HL_CXX_LIB=hdf5/hl/c++/src/.libs/libhdf5_hl_cpp.a
configure:    SZIP_LIB=hdf5/szip/src/.libs/libsz.a
configure: creating ./config.status
config.status: creating src/Makevars
** libs
mkdir -p "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"*.h "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/c++/src/"*.h "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/src/"*.h "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/c++/src/"*.h "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"libhdf5.settings "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
mkdir -p "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/src/.libs/libhdf5.a" "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp: cannot stat 'hdf5/src/.libs/libhdf5.a': No such file or directory
make: *** [copying] Error 1
ERROR: compilation failed for package 'Rhdf5lib'
* removing '/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/Rhdf5lib'
Error: Failed to install 'Rhdf5lib' from GitHub:
  (converted from warning) installation of package '/tmp/RtmpYhjnOU/filec777dc6cd74/Rhdf5lib_1.11.3.tar.gz' had non-zero exit status
Execution halted
✘ [Dec/05 17:22] vsc32884@tier2-p-login-1 ~ $   CC       H5HLdbg.lo
  CC       H5HLint.lo
  CC       H5HLprfx.lo
  CC       H5HLdblk.lo
  CC       H5HP.lo
  CC       H5I.lo
  CC       H5Itest.lo
  CC       H5L.lo
  CC       H5Lexternal.lo
  CC       H5make_libsettings.o
  CCLD     H5make_libsettings
/apps/leuven/skylake/2018a/software/binutils/2.28-GCCcore-6.4.0/bin/ld.gold: error: cannot find -lldap
/apps/leuven/skylake/2018a/software/binutils/2.28-GCCcore-6.4.0/bin/ld.gold: error: cannot find -llber
collect2: error: ld returned 1 exit status
make[2]: *** [H5make_libsettings] Error 1
make[2]: Leaving directory `/tmp/RtmpWh6BJm/R.INSTALL10401ea40216/Rhdf5lib/src/hdf5/src'
make[1]: *** [lib] Error 1
make[1]: Leaving directory `/tmp/RtmpWh6BJm/R.INSTALL10401ea40216/Rhdf5lib/src/hdf5/src'
make: *** [lib] Error 1
configure:    HDF5_INCLUDE=hdf5/src
configure:    HDF5_CXX_INCLUDE=hdf5/c++/src
configure:    HDF5_HL_INCLUDE=hdf5/hl/src
-bash: CC: command not foundc
onfigure:    HDF5_HL_CXX_INCLUDE=hdf5/hl/c++/src
configure:    HDF5_LIB=hdf5/src/.libs/libhdf5.a
configure:    HDF5_CXX_LIB=hdf5/c++/src/.libs/libhdf5_cpp.a
configure:    HDF5_HL_LIB=hdf5/hl/src/.libs/libhdf5_hl.a
configure:    HDF5_HL_CXX_LIB=hdf5/hl/c++/src/.libs/libhdf5_hl_cpp.a
configure:    SZIP_LIB=hdf5/szip/src/.libs/libsz.a
configure: creating ./config.status
config.status: creating src/Makevars
** libs
mkdir -p "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"*.h "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/c++/src/"*.h "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/src/"*.h "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/c++/src/"*.h "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"libhdf5.settings "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
mkdir -p "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/src/.libs/libhdf5.a" "/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp: cannot stat 'hdf5/src/.libs/libhdf5.a': No such file or directory
make: *** [copying] Error 1
ERROR: compilation failed for package 'Rhdf5lib'
* removing '/vsc-hard-mounts/leuven-user/328/vsc32884/Rlibs/Rhdf5lib'
Error: Failed to install 'Rhdf5lib' from GitHub:
  (converted from warning) installation of package '/tmp/RtmpYhjnOU/filec777dc6cd74/Rhdf5lib_1.11.3.tar.gz' had non-zero exit status
Execution halted

Is this a known issue/is there a known fix for this? Or maybe an implicit dependency that is missing?

grimbough commented 3 years ago

I've never seen this issue before, and I can't find anything in the HDF5 documentation that refers to those libraries. I presume installing libldap and liblber would be sufficient to solve the issue, but that doesn't help understanding why it's a problem in the first place. I don't know why it would be picking them up from, but they are installed on my test system, so perhaps it's being used implicitly as you say.

Can you give the output from running R CMD config LDFLAGS and R CMD config CPPFLAGS in a terminal?

There's also a section of the package installation that falls between the two lines below which would be useful to see.

SUMMARY OF THE HDF5 CONFIGURATION
=================================
...
<Lots of compilation settings>
...
Features:
---------
yentlprojects commented 3 years ago

Thanks for your fast reply.

Hereby the results of the commands you mentioned:

R CMD config LDFLAGS

-L/apps/leuven/skylake/2018a/software/GCCcore/6.4.0/lib64 -L/apps/leuven/skylake/2018a/software/GCCcore/6.4.0/lib -L/apps/leuven/skylake/2018a/software/OpenBLAS/0.2.20-GCC-6.4.0-2.28/lib -L/apps/leuven/skylake/2018a/software/FFTW/3.3.7-gompi-2018a/lib -L/apps/leuven/skylake/2018a/software/libreadline/7.0-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/ncurses/6.0-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/libpng/1.6.34-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/libjpeg-turbo/1.5.3-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/Java/1.8.0_162/lib -L/apps/leuven/skylake/2018a/software/ScaLAPACK/2.0.2-gompi-2018a-OpenBLAS-0.2.20/lib -L/apps/leuven/skylake/2018a/software/bzip2/1.0.6-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/cURL/7.58.0-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/PCRE2/10.21-foss-2018a/lib

R CMD config CPPFLAGS

-I/apps/leuven/skylake/2018a/software/OpenBLAS/0.2.20-GCC-6.4.0-2.28/include -I/apps/leuven/skylake/2018a/software/FFTW/3.3.7-gompi-2018a/include -I/apps/leuven/skylake/2018a/software/libreadline/7.0-GCCcore-6.4.0/include -I/apps/leuven/skylake/2018a/software/ncurses/6.0-GCCcore-6.4.0/include -I/apps/leuven/skylake/2018a/software/libpng/1.6.34-GCCcore-6.4.0/include -I/apps/leuven/skylake/2018a/software/libjpeg-turbo/1.5.3-GCCcore-6.4.0/include -I/apps/leuven/skylake/2018a/software/Java/1.8.0_162/include -I/apps/leuven/skylake/2018a/software/ScaLAPACK/2.0.2-gompi-2018a-OpenBLAS-0.2.20/include -I/apps/leuven/skylake/2018a/software/bzip2/1.0.6-GCCcore-6.4.0/include -I/apps/leuven/skylake/2018a/software/cURL/7.58.0-GCCcore-6.4.0/include -I/apps/leuven/skylake/2018a/software/PCRE2/10.21-foss-2018a/include

And also ran the installation again:

           SUMMARY OF THE HDF5 CONFIGURATION
            =================================

General Information:
-------------------
                   HDF5 Version: 1.10.6
                  Configured on: Mon Dec  7 20:06:11 CET 2020
                  Configured by: vsc32884@tier2-p-login-1
                    Host system: x86_64-unknown-linux-gnu
              Uname information: Linux tier2-p-login-1 3.10.0-1127.18.2.el7.x86_64 #1 SMP Sun Jul 26 15:27:06 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
                       Byte sex: little-endian
             Installation point: /tmp/RtmpqE2lRz/R.INSTALL41034aeb094/Rhdf5lib/src/hdf5/hdf5

Compiling Options:
------------------
                     Build Mode: production
              Debugging Symbols: no
                        Asserts: no
                      Profiling: no
             Optimization Level: high

Linking Options:
----------------
                      Libraries: static
  Statically Linked Executables:
                        LDFLAGS: -L/apps/leuven/skylake/2018a/software/GCCcore/6.4.0/lib64 -L/apps/leuven/skylake/2018a/software/GCCcore/6.4.0/lib -L/apps/leuven/skylake/2018a/software/OpenBLAS/0.2.20-GCC-6.4.0-2.28/lib -L/apps/leuven/skylake/2018a/software/FFTW/3.3.7-gompi-2018a/lib -L/apps/leuven/skylake/2018a/software/libreadline/7.0-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/ncurses/6.0-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/libpng/1.6.34-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/libjpeg-turbo/1.5.3-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/Java/1.8.0_162/lib -L/apps/leuven/skylake/2018a/software/ScaLAPACK/2.0.2-gompi-2018a-OpenBLAS-0.2.20/lib -L/apps/leuven/skylake/2018a/software/bzip2/1.0.6-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/cURL/7.58.0-GCCcore-6.4.0/lib -L/apps/leuven/skylake/2018a/software/PCRE2/10.21-foss-2018a/lib
                     H5_LDFLAGS:
                     AM_LDFLAGS:  -L/tmp/RtmpqE2lRz/R.INSTALL41034aeb094/Rhdf5lib/src/hdf5/szip/szip/lib
                Extra libraries: -lcrypto -lcurl -lsz -lz -ldl -lm
                       Archiver: ar
                       AR_FLAGS: cr
                         Ranlib: ranlib

Languages:
----------
                              C: yes
                     C Compiler: /apps/leuven/skylake/2018a/software/GCCcore/6.4.0/bin/gcc ( gcc (GCC) 6.4.0)
                       CPPFLAGS:
                    H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L   -DNDEBUG -UH5_DEBUG_API
                    AM_CPPFLAGS:  -I/tmp/RtmpqE2lRz/R.INSTALL41034aeb094/Rhdf5lib/src/hdf5/szip/szip/include
                        C Flags: -O2 -march=native -w -fpic
                     H5 C Flags:  -std=c99  -pedantic -Wall -Wextra -Wbad-function-cast -Wc++-compat -Wcast-align -Wcast-qual -Wconversion -Wdeclaration-after-statement -Wdisabled-optimization -Wfloat-equal -Wformat=2 -Wno-format-nonliteral -Winit-self -Winvalid-pch -Wmissing-declarations -Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-prototypes -Wswitch-enum -Wswitch-default -Wundef -Wunused-macros -Wunsafe-loop-optimizations -Wwrite-strings -Wlogical-op -Wlarger-than=2560 -Wsync-nand -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wstrict-overflow=5 -Wjump-misses-init -Wunsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wvector-operation-performance -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wnull-dereference -Wunused-const-variable -Wduplicated-cond -Whsa -Wnormalized -fstdarg-opt -s -Wno-inline -Wno-aggregate-return -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-suggest-attribute=const -Wno-suggest-attribute=pure -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=format -O3
                     AM C Flags:
               Shared C Library: no
               Static C Library: yes

                        Fortran: no

                            C++: yes
                   C++ Compiler: /apps/leuven/skylake/2018a/software/GCCcore/6.4.0/bin/g++ -std=gnu++11 ( g++ (GCC) 6.4.0)
                      C++ Flags: -O2 -march=native -w -fpic
                   H5 C++ Flags: -pedantic -Wall -Wextra -Wundef -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wconversion -Wredundant-decls -Winline -Wsign-promo -Woverloaded-virtual -Wold-style-cast -Weffc++ -Wreorder -Wnon-virtual-dtor -Wctor-dtor-privacy -Wabi -Wfloat-equal -Wmissing-format-attribute -Wmissing-noreturn -Wpacked -Wdisabled-optimization -Wformat=2 -Wendif-labels -Winvalid-pch -Wvariadic-macros -Winit-self -Wmissing-include-dirs -Wswitch-default -Wswitch-enum -Wunused-macros -Wunsafe-loop-optimizations -Wstrict-overflow -Wlogical-op -Wlarger-than=2048 -Wvla -Wsync-nand -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wstrict-overflow=5 -Wdouble-promotion -Wsuggest-attribute=const -Wtrampolines -Wstack-usage=8192 -Wvector-operation-performance -Wsuggest-attribute=pure -Wsuggest-attribute=noreturn -Wsuggest-attribute=format -Wdate-time -Wopenmp-simd -Warray-bounds=2 -fstdarg-opt -s -O3
                   AM C++ Flags:
             Shared C++ Library: no
             Static C++ Library: yes

                           Java: no

Features:
---------
                   Parallel HDF5: no
Parallel Filtered Dataset Writes: no
              Large Parallel I/O: no
              High-level library: yes
                Build HDF5 Tests: yes
                Build HDF5 Tools: yes
                    Threadsafety: no
             Default API mapping: v110
  With deprecated public symbols: yes
          I/O filters (external): deflate(zlib),szip(encoder)
                             MPE: no
                      Direct VFD: no
              (Read-Only) S3 VFD: yes
            (Read-Only) HDFS VFD: no
                         dmalloc: no
  Packages w/ extra debug output: none
                     API tracing: no
            Using memory checker: no
 Memory allocation sanity checks: no
          Function stack tracing: no
       Strict file format checks: no
    Optimization instrumentation: no

Thanks in advance for your assistance.

grimbough commented 3 years ago

Thanks for that. I notice that the CPPFLAGS are not being passed to the installer, so I've changed that. You can try testing via: 

BiocManager::install("grimbough/Rhdf5lib", ref = "cpp-flags")

I don't think that's going to be the solution though, as that should address finding header files rather than libraries. There's nothing obvious in the output that refers to libldap or libber, so presumably they're actually referenced by one of the other libraries. My guess would be libcurl, you can install a version of Rhdf5lib that doesn't use curl with:

BiocManager::install('grimbough/Rhdf5lib', ref="cpp-flags", configure.args = "--disable-ros3-vfd")

If that's still not working it might be good to run locate libldap and ldconfig -p | grep ldap to try and get an understanding of where that library is on your system.

yentlprojects commented 3 years ago

The first solution you proposed actually did the trick it seems. Managed to install Rhdf5lib (and flowCore) and get my scripts running!

Thanks a lot!

grimbough commented 3 years ago

Great, that was definitely an oversight in the installation process and needed to be fixed, but I wasn't expecting it to work here! Glad it is. I'll merge the change into the devel and release versions of the package in Bioconductor, so you should be able to install using the standard mechanism going forward.

grimbough commented 3 years ago

This patch should now be in both release (>= 1.12.1) and devel (>= 1.13.2). Please reopen the issue if you encounter it again.

ColeWunderlich commented 3 years ago

I am also getting the same error on CentOS 8 with R 4.0.2 and GCC 8.3.0

Looks like it is related to the gold linker and GCC.

Abbreviated output below (full dump attached)

        SUMMARY OF THE HDF5 CONFIGURATION
        =================================

General Information:
-------------------
                   HDF5 Version: 1.10.6
                  Configured on: Tue Feb 23 00:21:49 EST 2021
                  Configured by: wunderl@bamdev1
                    Host system: x86_64-unknown-linux-gnu
              Uname information: Linux bamdev1 4.18.0-193.14.2.el8_2.x86_64 #1 SMP Sun Jul 26 03:54:29 UTC 2020 x86_64 x86_64 x86_64 GNU/Linux
                       Byte sex: little-endian
             Installation point: /tmp/Rtmp0PHxII/R.INSTALL3d4e859dace7c/Rhdf5lib/src/hdf5/hdf5

Compiling Options:
------------------
                     Build Mode: production
              Debugging Symbols: no
                        Asserts: no
                      Profiling: no
             Optimization Level: high

Linking Options:
----------------
                      Libraries: static
  Statically Linked Executables: 
                        LDFLAGS: -L/grid/it/data/elzar/easybuild/software/GLPK/4.65-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/ImageMagick/7.0.9-5-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/GSL/2.6-GCC-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/UDUNITS/2.2.26-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/HDF5/1.10.5-gompi-2019b/lib -L/grid/it/data/elzar/easybuild/software/ICU/64.2-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libsndfile/1.0.28-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/FFTW/3.3.8-gompi-2019b/lib -L/grid/it/data/elzar/easybuild/software/NLopt/2.6.1-GCCcore-8.3.0/lib64 -L/grid/it/data/elzar/easybuild/software/GMP/6.1.2-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libxml2/2.9.9-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/cURL/7.66.0-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/Tk/8.6.9-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/Java/11.0.2/lib -L/grid/it/data/elzar/easybuild/software/LibTIFF/4.0.10-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libjpeg-turbo/2.0.3-GCCcore-8.3.0/lib64 -L/grid/it/data/elzar/easybuild/software/libpng/1.6.37-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/PCRE/8.43-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/SQLite/3.29.0-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/zlib/1.2.11-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/XZ/5.2.4-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/bzip2/1.0.8-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/ncurses/6.1-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libreadline/8.0-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/cairo/1.16.0-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libGLU/9.0.1-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/Mesa/19.1.7-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/X11/20190717-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/ScaLAPACK/2.0.2-gompi-2019b/lib -L/grid/it/data/elzar/easybuild/software/OpenBLAS/0.3.7-GCC-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/GCCcore/8.3.0/lib64 -L/grid/it/data/elzar/easybuild/software/GCCcore/8.3.0/lib
                     H5_LDFLAGS: 
                     AM_LDFLAGS:  -L/tmp/Rtmp0PHxII/R.INSTALL3d4e859dace7c/Rhdf5lib/src/hdf5/szip/szip/lib
                Extra libraries: -lcrypto -lcurl -lsz -lz -ldl -lm 
                       Archiver: ar
                       AR_FLAGS: cr
                         Ranlib: ranlib

Languages:
----------
                              C: yes
                     C Compiler: /grid/it/data/elzar/easybuild/software/GCCcore/8.3.0/bin/gcc ( gcc (GCC) 8.3.0)
                       CPPFLAGS: 
                    H5_CPPFLAGS: -D_GNU_SOURCE -D_POSIX_C_SOURCE=200809L   -DNDEBUG -UH5_DEBUG_API
                    AM_CPPFLAGS:  -I/tmp/Rtmp0PHxII/R.INSTALL3d4e859dace7c/Rhdf5lib/src/hdf5/szip/szip/include
                        C Flags: -O2 -ftree-vectorize -march=native -fno-math-errno -w -fpic
                     H5 C Flags:  -std=c99  -pedantic -Wall -Wextra -Wbad-function-cast -Wc++-compat -Wcast-align -Wcast-qual -Wconversion -Wdeclaration-after-statement -Wdisabled-optimization -Wfloat-equal -Wformat=2 -Wno-format-nonliteral -Winit-self -Winvalid-pch -Wmissing-declarations -Wmissing-include-dirs -Wmissing-prototypes -Wnested-externs -Wold-style-definition -Wpacked -Wredundant-decls -Wshadow -Wstrict-prototypes -Wswitch-enum -Wswitch-default -Wundef -Wunused-macros -Wunsafe-loop-optimizations -Wwrite-strings -Wlogical-op -Wlarger-than=2560 -Wsync-nand -Wframe-larger-than=16384 -Wpacked-bitfield-compat -Wstrict-overflow=5 -Wjump-misses-init -Wunsuffixed-float-constants -Wdouble-promotion -Wtrampolines -Wstack-usage=8192 -Wvector-operation-performance -Wdate-time -Warray-bounds=2 -Wc99-c11-compat -Wnull-dereference -Wunused-const-variable -Wduplicated-cond -Whsa -Wnormalized -Walloc-zero -Walloca -Wduplicated-branches -Wformat-overflow=2 -Wformat-truncation=2 -Wimplicit-fallthrough=5 -Wrestrict -Wattribute-alias -Wcast-align=strict -Wshift-overflow=2 -fstdarg-opt -s -Wno-inline -Wno-aggregate-return -Wno-missing-format-attribute -Wno-missing-noreturn -Wno-suggest-attribute=const -Wno-suggest-attribute=pure -Wno-suggest-attribute=noreturn -Wno-suggest-attribute=format -Wno-suggest-attribute=cold -Wno-suggest-attribute=malloc -O3
                     AM C Flags: 
               Shared C Library: no
               Static C Library: yes

                        Fortran: no

                            C++: yes
                   C++ Compiler: /grid/it/data/elzar/easybuild/software/GCCcore/8.3.0/bin/g++ -std=gnu++11 ( g++ (GCC) 8.3.0)
                      C++ Flags: -O2 -ftree-vectorize -march=native -fno-math-errno -w -fpic
                   H5 C++ Flags:   -pedantic -Wall -W -Wundef -Wshadow -Wpointer-arith -Wcast-qual -Wcast-align -Wwrite-strings -Wconversion -Wredundant-decls -Winline -Wsign-promo -Woverloaded-virtual -Wold-style-cast -Weffc++ -Wreorder -Wnon-virtual-dtor -Wctor-dtor-privacy -Wabi -finline-functions -s -O
                   AM C++ Flags: 
             Shared C++ Library: no
             Static C++ Library: yes

                           Java: no

Features:
---------
                   Parallel HDF5: no
Parallel Filtered Dataset Writes: no
              Large Parallel I/O: no
              High-level library: yes
                Build HDF5 Tests: yes
                Build HDF5 Tools: yes
                    Threadsafety: no
             Default API mapping: v110
  With deprecated public symbols: yes
          I/O filters (external): deflate(zlib),szip(encoder)
                             MPE: no
                      Direct VFD: no
              (Read-Only) S3 VFD: yes
            (Read-Only) HDFS VFD: no
                         dmalloc: no
  Packages w/ extra debug output: none
                     API tracing: no
            Using memory checker: no
 Memory allocation sanity checks: no
          Function stack tracing: no
       Strict file format checks: no
    Optimization instrumentation: no
for d in src . c++ hl; do                                \
  if test $d != .; then                                        \
   (cd $d && make  lib) || exit 1;            \
  fi;                                                           \
done
make[1]: Entering directory '/tmp/Rtmp0PHxII/R.INSTALL3d4e859dace7c/Rhdf5lib/src/hdf5/src'
make[2]: Entering directory '/tmp/Rtmp0PHxII/R.INSTALL3d4e859dace7c/Rhdf5lib/src/hdf5/src'
..........
  CC       H5Itest.lo
  CC       H5L.lo
  CC       H5Lexternal.lo
  CC       H5make_libsettings.o
  CCLD     H5make_libsettings
/grid/it/data/elzar/easybuild/software/binutils/2.32-GCCcore-8.3.0/bin/ld.gold: error: cannot find -lldap
/grid/it/data/elzar/easybuild/software/binutils/2.32-GCCcore-8.3.0/bin/ld.gold: error: cannot find -llber
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:1122: H5make_libsettings] Error 1
make[2]: Leaving directory '/tmp/Rtmp0PHxII/R.INSTALL3d4e859dace7c/Rhdf5lib/src/hdf5/src'
make[1]: *** [Makefile:1989: lib] Error 1
make[1]: Leaving directory '/tmp/Rtmp0PHxII/R.INSTALL3d4e859dace7c/Rhdf5lib/src/hdf5/src'
make: *** [Makefile:1088: lib] Error 1
configure:    HDF5_INCLUDE=hdf5/src
configure:    HDF5_CXX_INCLUDE=hdf5/c++/src
configure:    HDF5_HL_INCLUDE=hdf5/hl/src
configure:    HDF5_HL_CXX_INCLUDE=hdf5/hl/c++/src
configure:    HDF5_LIB=hdf5/src/.libs/libhdf5.a
configure:    HDF5_CXX_LIB=hdf5/c++/src/.libs/libhdf5_cpp.a
configure:    HDF5_HL_LIB=hdf5/hl/src/.libs/libhdf5_hl.a
configure:    HDF5_HL_CXX_LIB=hdf5/hl/c++/src/.libs/libhdf5_hl_cpp.a
configure:    SZIP_LIB=hdf5/szip/src/.libs/libsz.a
configure: creating ./config.status
config.status: creating src/Makevars
** libs
mkdir -p "/grid/wsbs/home_norepl/wunderl/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"*.h "/grid/wsbs/home_norepl/wunderl/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/c++/src/"*.h "/grid/wsbs/home_norepl/wunderl/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/src/"*.h "/grid/wsbs/home_norepl/wunderl/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/c++/src/"*.h "/grid/wsbs/home_norepl/wunderl/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
mkdir -p "/grid/wsbs/home_norepl/wunderl/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/src/.libs/libhdf5.a" "/grid/wsbs/home_norepl/wunderl/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp: cannot stat 'hdf5/src/.libs/libhdf5.a': No such file or directory
make: *** [Makevars:38: copying] Error 1
ERROR: compilation failed for package ‘Rhdf5lib’

hdf5lib_errorDump.txt

ColeWunderlich commented 3 years ago

Update: I just realized the latest version available through Bioconductor for R 4.0.2 is Rhdf5lib 1.12.1 which does not contain the patch mentioned here.

Installing from github@master failed as it requires R>4.1.0. Installing from github@cpp-flags , however, worked!

I don't know if it's possible, but it would be nice to update the package on Bioconductor 3.12 to include the cpp-flags patch, as it appears the current v1.12.1 lacks the patch causing it to break on CentOS.

grimbough commented 3 years ago

Thanks for the extra info. I thought I'd merged that cpp-flag patch into the release version, but perhaps not. I thought that the issue was not passing CPP_FLAGS during compilation, but I see them in the build reports for 1.12.1 e.g. CPPFLAGS: -I/usr/local/include in http://bioconductor.org/checkResults/release/bioc-LATEST/Rhdf5lib/nebbiolo1-install.html

Does it also fail if you installed the RELEASE_3_12 branch from here?

BiocManager::install("grimbough/Rhdf5lib", ref = "RELEASE_3_12")

That should be identical to the Bioconductor 3.12 version (assuming I've pushed to all the correct places). Maybe also confirm you're definitely using Bioconductor 3.12 with BiocManager::version() ?

ColeWunderlich commented 3 years ago

With respect to the build report, it looks like fundamentally different code is being run, perhaps a shell script designed to handle this exception that is not currently included in 1.12.1.

The relevant bit from the build report

  CC       H5Lexternal.lo
  CC       H5make_libsettings.o
  CCLD     H5make_libsettings
LD_LIBRARY_PATH="$LD_LIBRARY_PATH`echo -L/usr/local/lib |                  \
    sed -e 's/-L/:/g' -e 's/ //g'`"                               \
 ./H5make_libsettings > H5lib_settings.c  ||                               \
    (test $HDF5_Make_Ignore && echo "*** Error ignored") ||          \
    (rm -f H5lib_settings.c ; exit 1)
  CC       H5lib_settings.lo

But on my system

  CC       H5Lexternal.lo
  CC       H5make_libsettings.o
  CCLD     H5make_libsettings
/grid/it/data/elzar/easybuild/software/binutils/2.32-GCCcore-8.3.0/bin/ld.gold: error: cannot find -lldap
/grid/it/data/elzar/easybuild/software/binutils/2.32-GCCcore-8.3.0/bin/ld.gold: error: cannot find -llber
collect2: error: ld returned 1 exit status
make[2]: *** [Makefile:1122: H5make_libsettings] Error 1

It is also true, however, that my system does not have the cpp flags listed and they are listed in the build report.

As for the other questions:

later

CC H5make_libsettings.o CCLD H5make_libsettings /grid/it/data/elzar/easybuild/software/binutils/2.32-GCCcore-8.3.0/bin/ld.gold: error: cannot find -lldap /grid/it/data/elzar/easybuild/software/binutils/2.32-GCCcore-8.3.0/bin/ld.gold: error: cannot find -llber collect2: error: ld returned 1 exit status


- Apparently I am actually on Bioconductor version `3.11` as `3.12` is only supported for `R 4.0.3` and up.  
- Downloading the `v1.12.1` tarball from `Bioc 13.12` and installing from source yields the same error. The `1.12.1` install does show the cpp flags, however, and looks the same as the output from `RELEASE_3_12`
- Installing `v1.10.1` from `Bioc 3.11` also yields the same error 

Overall it looks like it is not actually the missing `CPPFLAGS` that is causing the error, but rather something the `cpp-flags` branch has that `RELEASE_3_12` lacks.  Comparing the two branches on github, however,  I do see a few places `RELEASE_3_12` lacks the cpp flags.

[In this diff](https://github.com/grimbough/Rhdf5lib/compare/RELEASE_3_12...cpp-flags#diff-90d08e583c4c9c6f391b2ae90f819f600a6326928ea9512c9e0c6d98e9f29ac2R2158) lines 2158, 3890, and 3903 in `configure` are missing  `CPPFLAGS`.  Lines 23, 116, and 129 in `configure.ac` are also missing the flags.
grimbough commented 3 years ago

Thanks for the detective work. I think what must have happened is that I merged the cpp-branch into the Bioconductor devel branch, but then just tried to apply the changes manually to RELEASE_3_12. I guess I didn't do that successfully! I'll apply the missing parts to the branch and create version 1.12.2.

I'm only able to do this for 3.12. The 3.11 branch is locked now at Bioconductor, so I'm afraid that will never be patched.

Also, despite what it says at https://www.bioconductor.org/news/bioc_3_12_release/ about being tied to R-4.0.3 I think you should be fine to use any version of R-4.0.Z. AFAIK Bioconductor versions are tied to the Y parts of R-X.Y.Z, but not the point releases.

grimbough commented 3 years ago

So I'm even more confused by this. Those CPPFLAGS statements are present in the RELEASE_3_12 branch and on Bioconductor e.g.

https://github.com/grimbough/Rhdf5lib/blob/cf464f40fd95274d0d351cf28b586c49307c4f0b/configure.ac#L121

I'm not sure exactly what the diff is showing, presumably its relative to some common ancestor rather than the current state of the RELEASE_3_12 branch. That explains why you see the CPPFLAGS in the configuration table with RELEASE_3_12 but doesn't help us know where those missing libraries should be found.

grimbough commented 3 years ago

After some more digging it seem that the cpp-flags branch also disables the linking with libcurl, which in turn means it doesn't look for the two libraries it can't find on your system. Success by accident unfortunately.

On the CentOS / Easybuild system I have access to the version of cURL does not provide support for the ldap protocol or link to the library. This is presumably why the versions of Rhf5lib in Bioconductor install fine for me on that system.

% curl --version
curl 7.66.0 (x86_64-pc-linux-gnu) libcurl/7.66.0 OpenSSL/1.0.2k-fips zlib/1.2.11
Release-Date: 2019-09-11
Protocols: dict file ftp ftps gopher http https imap imaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL UnixSockets

% curl-config --static-libs            
/g/easybuild/x86_64/CentOS/7/haswell/software/cURL/7.66.0-GCCcore-8.3.0/lib/libcurl.a 
-L/g/easybuild/x86_64/CentOS/7/haswell/software/zlib/1.2.11-GCCcore-8.3.0/lib 
-L/g/easybuild/x86_64/CentOS/7/haswell/software/binutils/2.32-GCCcore-8.3.0/lib 
-L/g/easybuild/x86_64/CentOS/7/haswell/software/GCCcore/8.3.0/lib64 
-L/g/easybuild/x86_64/CentOS/7/haswell/software/GCCcore/8.3.0/lib 
-L/g/easybuild/x86_64/CentOS/7/haswell/software/zlib/1.2.11-GCCcore-8.3.0/lib -lssl -lcrypto -lssl -lcrypto -lz -lm -lpthread

I assume that if you run those two commands on your system it will mention ldap on the "protocols" line as well as -lldap and -llber in the last part.

grimbough commented 3 years ago

Hopefully I've fixed up the configure logic in a new branch TEST_3_12

I expect this should failed as normal:

BiocManager::install('grimbough/Rhdf5lib', ref='TEST_3_12')

But this should at least install:

BiocManager::install('grimbough/Rhdf5lib', ref='TEST_3_12', configure.args = '--disable-ros3-vfd')
ColeWunderlich commented 3 years ago

Now that you mention it, there was something really odd about that diff. I tried reversing it (ie diff TEST_3_12 cpp-flags vs diff cpp-flags RELEASE_3_12) and the results were not symmetrical. I thought maybe I was just missing something but now I'm starting to think there must be some quirk on Github's end.

Installation attempts

I tried the version with configure.args = '--disable-ros3-vfd' and it installed successfully.

Output

CC       H5make_libsettings.o
  CCLD     H5make_libsettings
LD_LIBRARY_PATH="$LD_LIBRARY_PATH`echo -L/grid/it/data/elzar/easybuild/software/GLPK/4.65-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/ImageMagick/7.0.9-5-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/GSL/2.6-GCC-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/UDUNITS/2.2.26-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/HDF5/1.10.5-gompi-2019b/lib -L/grid/it/data/elzar/easybuild/software/ICU/64.2-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libsndfile/1.0.28-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/FFTW/3.3.8-gompi-2019b/lib -L/grid/it/data/elzar/easybuild/software/NLopt/2.6.1-GCCcore-8.3.0/lib64 -L/grid/it/data/elzar/easybuild/software/GMP/6.1.2-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libxml2/2.9.9-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/cURL/7.66.0-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/Tk/8.6.9-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/Java/11.0.2/lib -L/grid/it/data/elzar/easybuild/software/LibTIFF/4.0.10-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libjpeg-turbo/2.0.3-GCCcore-8.3.0/lib64 -L/grid/it/data/elzar/easybuild/software/libpng/1.6.37-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/PCRE/8.43-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/SQLite/3.29.0-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/zlib/1.2.11-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/XZ/5.2.4-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/bzip2/1.0.8-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/ncurses/6.1-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libreadline/8.0-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/cairo/1.16.0-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/libGLU/9.0.1-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/Mesa/19.1.7-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/X11/20190717-GCCcore-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/ScaLAPACK/2.0.2-gompi-2019b/lib -L/grid/it/data/elzar/easybuild/software/OpenBLAS/0.3.7-GCC-8.3.0/lib -L/grid/it/data/elzar/easybuild/software/GCCcore/8.3.0/lib64 -L/grid/it/data/elzar/easybuild/software/GCCcore/8.3.0/lib |                  \
    sed -e 's/-L/:/g' -e 's/ //g'`"                               \
 ./H5make_libsettings > H5lib_settings.c  ||                               \
    (test $HDF5_Make_Ignore && echo "*** Error ignored") ||          \
    (rm -f H5lib_settings.c ; exit 1)
  CC       H5lib_settings.lo

#but then later...
CXX      H5PacketTable.lo
  CXXLD    libhdf5_hl_cpp.la
libtool: warning: '/grid/it/data/elzar/easybuild/software/GCCcore/8.3.0/lib64/libstdc++.la' seems to be moved
make[4]: Leaving directory '/tmp/RtmpuT239e/R.INSTALL2d4258696a8473/Rhdf5lib/src/hdf5/hl/c++/src'
make[3]: Leaving directory '/tmp/RtmpuT239e/R.INSTALL2d4258696a8473/Rhdf5lib/src/hdf5/hl/c++/src'

I do get the one warning, but the package appears to work fine.

Installing without the configure.args = '--disable-ros3-vfd' fails as you predicted, giving the same error as before.

Curl output

Your guess is correct. I have the ldap protocol and -lldap -llber on the last lib line.

$curl --version

curl 7.66.0 (x86_64-pc-linux-gnu) libcurl/7.66.0 OpenSSL/1.1.1g-fips zlib/1.2.11
Release-Date: 2019-09-11
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS HTTPS-proxy IPv6 Largefile libz NTLM NTLM_WB SSL TLS-SRP UnixSockets
$ curl-config --static-libs
/grid/it/data/elzar/easybuild/software/cURL/7.66.0-GCCcore-8.3.0/lib/libcurl.a 
-L/grid/it/data/elzar/easybuild/software/zlib/1.2.11-GCCcore-8.3.0/lib 
-L/grid/it/data/elzar/easybuild/software/binutils/2.32-GCCcore-8.3.0/lib 
-L/grid/it/data/elzar/easybuild/software/GCCcore/8.3.0/lib64 
-L/grid/it/data/elzar/easybuild/software/GCCcore/8.3.0/lib -lssl -lcrypto -lssl -lcrypto -lldap -llber -lz -lm -lpthread

What may also be of interest:

$ ldd /grid/it/data/elzar/easybuild/software/cURL/7.66.0-GCCcore-8.3.0/bin/curl
    linux-vdso.so.1 (0x0000155555551000)
    libcurl.so.4 => /grid/it/data/elzar/easybuild/software/cURL/7.66.0-GCCcore-8.3.0/lib/libcurl.so.4 (0x00001555554cd000)
    libssl.so.1.1 => /lib64/libssl.so.1.1 (0x0000155555097000)
    libcrypto.so.1.1 => /lib64/libcrypto.so.1.1 (0x0000155554bb1000)
    libz.so.1 => /grid/it/data/elzar/easybuild/software/zlib/1.2.11-GCCcore-8.3.0/lib/libz.so.1 (0x000015555549b000)
    libm.so.6 => /lib64/libm.so.6 (0x000015555482f000)
    libpthread.so.0 => /lib64/libpthread.so.0 (0x000015555460f000)
    libc.so.6 => /lib64/libc.so.6 (0x000015555424c000)
    libldap-2.4.so.2 => /lib64/libldap-2.4.so.2 (0x0000155553ffe000)
    liblber-2.4.so.2 => /lib64/liblber-2.4.so.2 (0x0000155553dee000)
    libdl.so.2 => /lib64/libdl.so.2 (0x0000155553bea000)
    /lib64/ld-linux-x86-64.so.2 (0x000015555532b000)
    libresolv.so.2 => /lib64/libresolv.so.2 (0x00001555539d3000)
    libsasl2.so.3 => /lib64/libsasl2.so.3 (0x00001555537b5000)
    libcrypt.so.1 => /lib64/libcrypt.so.1 (0x000015555358c000)
    libgssapi_krb5.so.2 => /lib64/libgssapi_krb5.so.2 (0x0000155553337000)
    libkrb5.so.3 => /lib64/libkrb5.so.3 (0x000015555304e000)
    libk5crypto.so.3 => /lib64/libk5crypto.so.3 (0x0000155552e37000)
    libcom_err.so.2 => /lib64/libcom_err.so.2 (0x0000155552c33000)
    libkrb5support.so.0 => /lib64/libkrb5support.so.0 (0x0000155552a22000)
    libkeyutils.so.1 => /lib64/libkeyutils.so.1 (0x000015555281e000)
    libselinux.so.1 => /lib64/libselinux.so.1 (0x00001555525f4000)
    libpcre2-8.so.0 => /lib64/libpcre2-8.so.0 (0x0000155552370000)

Following suggestions from https://github.com/rust-lang/cargo/issues/684 and https://unix.stackexchange.com/questions/569385/ld-cannot-find-library-right-in-front-of-it it appears that the issue may be that static linking is expecting only the names liblber.so and libldap.so so the versioned names (eg libldap-2.4.so.2) are causing the linker to complain that they cannot be found. This seems to be supported by the fact that this issue tends to be resolved when the dev versions are installed (which I am assuming don't have shared objects with versioned names).

I would create symlinks without the version names to test this, but I regrettably am not an admin on my server so I don't have write access to /lib64/ (which is really /usr/lib64)

grimbough commented 3 years ago

Thanks for confirming that. It seems like there's definitely something a bit odd with the configuration of cURL on your system, but at least you can get an version installed by providing that argument. However cURL is not essential to using Rhdf5lib, so it'd be nice if the install could handle this more gracefully and get the --disable--ros3-vfd part automatically.

The configure.ac script is supposed to test the linking step automatically, but that clearly doesn't seem to be quite doing it correctly. I've now put a minimal working C programming in the configuration, that should require linking to work successfully. It's be great if you could try installing from that test branch again:

BiocManager::install('grimbough/Rhdf5lib', ref='TEST_3_12

Near the top of the output there should be a series of # that make the new output. It'd be great if you could share the output from that. Here's the output I see, I'm hoping yours says ...no as then the test is replicating the real problem we see later.

   checking for library containing curl_global_init... -lcurl
   checking for library containing EVP_sha256... -lcrypto
   #######################################
   -lcrypto -lcurl
   checking if linking to libcurl works... yes
   #######################################
   S3_VFD=--enable-ros3-vfd
ColeWunderlich commented 3 years ago

I tried the test branch again and the install failed with the same error as before.

Here is the output from the area you requested. Unfortunately it says yes :/

checking openssl/sha.h usability... yes
checking openssl/sha.h presence... yes
checking for openssl/sha.h... yes
checking for library containing curl_global_init... -lcurl
checking for library containing EVP_sha256... -lcrypto
#######################################
-lcrypto -lcurl
checking if linking to libcurl works... yes
#######################################
S3_VFD=--enable-ros3-vfd
untarring hdf5small_cxx_hl_1.10.6.tar.gz ...
building the szip library...
checking for a BSD-compatible install... /usr/bin/install -c
lyj95618 commented 3 years ago

Hello,

I encountered a similar issue when I tried to install HDF5Array which depends on Rhd5lib. I am using R 4.0.3, Bioconductor version 3.12, Centos7.

Here is part of the output

/bin/ld: cannot find -lldap
collect2: error: ld returned 1 exit status
make[2]: *** [H5make_libsettings] Error 1
make[2]: Leaving directory `/localhd/tmp/Rtmp8p1Tlt/R.INSTALL101ab723d5bfb/Rhdf5lib/src/hdf5/src'
make[1]: *** [lib] Error 1
make[1]: Leaving directory `/localhd/tmp/Rtmp8p1Tlt/R.INSTALL101ab723d5bfb/Rhdf5lib/src/hdf5/src'
make: *** [lib] Error 1
configure:    HDF5_INCLUDE=hdf5/src
configure:    HDF5_CXX_INCLUDE=hdf5/c++/src
configure:    HDF5_HL_INCLUDE=hdf5/hl/src
configure:    HDF5_HL_CXX_INCLUDE=hdf5/hl/c++/src
configure:    HDF5_LIB=hdf5/src/.libs/libhdf5.a
configure:    HDF5_CXX_LIB=hdf5/c++/src/.libs/libhdf5_cpp.a
configure:    HDF5_HL_LIB=hdf5/hl/src/.libs/libhdf5_hl.a
configure:    HDF5_HL_CXX_LIB=hdf5/hl/c++/src/.libs/libhdf5_hl_cpp.a
configure:    SZIP_LIB=hdf5/szip/src/.libs/libsz.a
configure: creating ./config.status
config.status: creating src/Makevars
** libs
mkdir -p "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"*.h "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/c++/src/"*.h "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/src/"*.h "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/c++/src/"*.h "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"libhdf5.settings "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
mkdir -p "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/src/.libs/libhdf5.a" "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp: cannot stat ‘hdf5/src/.libs/libhdf5.a’: No such file or directory
make: *** [copying] Error 1
ERROR: compilation failed for package ‘Rhdf5lib’
* removing ‘/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib’
ERROR: dependency ‘Rhdf5lib’ is not available for package ‘rhdf5filters’
* removing ‘/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/rhdf5filters’
ERROR: dependencies ‘Rhdf5lib’, ‘rhdf5filters’ are not available for package ‘rhdf5’
* removing ‘/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/rhdf5’
ERROR: dependencies ‘rhdf5’, ‘Rhdf5lib’ are not available for package ‘HDF5Array’
* removing ‘/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/HDF5Array’
grimbough commented 3 years ago

@lyj95618 Is this still an issue if you install the test branch from here with the following command?

BiocManager::install('grimbough/Rhdf5lib', ref='TEST_3_12')

It would also be good if you could share the part near the top of the output where there's a bunch of #. Here's the output I see for that part:

checking for library containing curl_global_init... -lcurl
checking for library containing EVP_sha256... -lcrypto
#######################################
-lcrypto -lcurl
checking if linking to libcurl works... yes
#######################################
S3_VFD=--enable-ros3-vfd
lyj95618 commented 3 years ago

Hello, thanks for the quick reply!

The issue still exists after I ran BiocManager::install('grimbough/Rhdf5lib', ref='TEST_3_12')

These are the parts of the output from this command

checking for openssl/sha.h... yes
checking for library containing curl_global_init... -lcurl
checking for library containing EVP_sha256... -lcrypto
#######################################
-lcrypto -lcurl
checking if linking to libcurl works... yes
#######################################
S3_VFD=--enable-ros3-vfd
untarring hdf5small_cxx_hl_1.10.6.tar.gz ...
/bin/ld: cannot find -lldap
collect2: error: ld returned 1 exit status
make[2]: *** [H5make_libsettings] Error 1
make[2]: Leaving directory `/localhd/tmp/RtmpqWf4VX/R.INSTALLf2b8124e7c9/Rhdf5lib/src/hdf5/src'
make[1]: *** [lib] Error 1
make[1]: Leaving directory `/localhd/tmp/RtmpqWf4VX/R.INSTALLf2b8124e7c9/Rhdf5lib/src/hdf5/src'
make: *** [lib] Error 1
configure:    HDF5_INCLUDE=hdf5/src
configure:    HDF5_CXX_INCLUDE=hdf5/c++/src
configure:    HDF5_HL_INCLUDE=hdf5/hl/src
configure:    HDF5_HL_CXX_INCLUDE=hdf5/hl/c++/src
configure:    HDF5_LIB=hdf5/src/.libs/libhdf5.a
configure:    HDF5_CXX_LIB=hdf5/c++/src/.libs/libhdf5_cpp.a
configure:    HDF5_HL_LIB=hdf5/hl/src/.libs/libhdf5_hl.a
configure:    HDF5_HL_CXX_LIB=hdf5/hl/c++/src/.libs/libhdf5_hl_cpp.a
configure:    SZIP_LIB=hdf5/szip/src/.libs/libsz.a
configure: creating ./config.status
config.status: creating src/Makevars
** libs
mkdir -p "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"*.h "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/c++/src/"*.h "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/src/"*.h "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/hl/c++/src/"*.h "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
cp "hdf5/src/"libhdf5.settings "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/include"
mkdir -p "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp "hdf5/src/.libs/libhdf5.a" "/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-Rhdf5lib/00new/Rhdf5lib/lib/"
cp: cannot stat ‘hdf5/src/.libs/libhdf5.a’: No such file or directory
make: *** [copying] Error 1
ERROR: compilation failed for package ‘Rhdf5lib’
* removing ‘/home/yliang/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib’
Error: Failed to install 'Rhdf5lib' from GitHub:
  (converted from warning) installation of package ‘/tmp/RtmpxBPaLf/fileb5146940fb7/Rhdf5lib_1.12.1-1.tar.gz’ had non-zero exit status

I managed to install the package using this command (I dont see the output with a bunch of # this time). Things are working now! BiocManager::install('grimbough/Rhdf5lib', ref='TEST_3_12', configure.args = '--disable-ros3-vfd')

Thank you very much!

ScottNortonPhD commented 2 years ago

Issue persists on Bioconductor upstream, which is problematic as trying to install glmGamPoi (which depends on Rhdf5lib and others) will automatically select the latest version of Rhdf5lib. Can these changes be reimplemented on the current patch?

grimbough commented 2 years ago

Issue persists on Bioconductor upstream, which is problematic as trying to install glmGamPoi (which depends on Rhdf5lib and others) will automatically select the latest version of Rhdf5lib. Can these changes be reimplemented on the current patch?

Does running BiocManager::install("Rhdf5lib", configure.args = '--disable-ros3-vfd') help? I thought I'd made that option available in both release and devel branches for a while now, but maybe not.

vinh-tr commented 1 year ago

Hello, I got the same error when running BiocManager::install("Rhdf5lib", configure.args = '--disable-ros3-vfd'). The option is not recognized : configure: WARNING: unrecognized options: --disable-ros3-vfd

The following lines are part of the error message: /cm/easybuild/software/binutils/2.35-GCCcore-10.2.0/bin/ld.gold: error: cannot find -lldap /cm/easybuild/software/binutils/2.35-GCCcore-10.2.0/bin/ld.gold: error: cannot find -llber

Best

jd-daniels commented 1 year ago

Can confirm that I only found that option "--disable-ros3-vfd" available on that TEST_3_12. Any newer or latest refs don't accept that and fail with ldap/lber errors. However when I use that TEST_3_12 everything works fine, besides it being an old version now. Any chance those can be made available on the latest iterations?

R 4.2.3. RHEL 9.1.

Thanks, John

mehtar1 commented 1 year ago

I tried using TEST_3_12 along with the option "--disable_ros3-vfd" to prevent the same error, and I receive this message: 

installing to /home/mehtar1/R/x86_64-pc-linux-gnu-library/4.2/00LOCK-Rhdf5lib/00new/Rhdf5lib/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
** checking absolute paths in shared objects and dynamic libraries
** testing if installed package can be loaded from final location
** testing if installed package keeps a record of temporary installation path
* DONE (Rhdf5lib)
Installation paths not writeable, unable to update packages
  path: /cm/shared/apps_chop/R/4.2.3/lib64/R/library
  packages:
    knitr, Matrix, s2, scattermore, sf, waldo
Old packages: 'Rhdf5lib'

It prompts me to update the package at which point I run into the same issue. Is there any way to avoid this?