undefined symbol: H5T_NATIVE_INT32_g

[descostesn]

Hi,

I am on ubuntu 18.04.4 LTS and using R 4.0.0. When trying to install the package I get:

> BiocManager::install("rhdf5")
Bioconductor version 3.11 (BiocManager 1.30.10), R 4.0.0 (2020-04-24)
Installing package(s) 'rhdf5'
trying URL 'https://bioconductor.org/packages/3.11/bioc/src/contrib/rhdf5_2.32.4.tar.gz'
Content type 'application/x-gzip' length 838257 bytes (818 KB)
==================================================
downloaded 818 KB

* installing *source* package ‘rhdf5’ ...
** using staged installation
** libs
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5.c -o H5.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5A.c -o H5A.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5D.c -o H5D.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5E.c -o H5E.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5F.c -o H5F.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5G.c -o H5G.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5I.c -o H5I.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5L.c -o H5L.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5O.c -o H5O.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5P.c -o H5P.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5S.c -o H5S.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5T.c -o H5T.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5Z.c -o H5Z.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c H5constants.c -o H5constants.o
g++ -std=gnu++11 -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c HandleList.cpp -o HandleList.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c HandleListWrap.c -o HandleListWrap.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c h5dump.c -o h5dump.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c h5ls.c -o h5ls.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c h5testLock.c -o h5testLock.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c h5writeDataFrame.c -o h5writeDataFrame.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c printdatatype.c -o printdatatype.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c utils.c -o utils.o
gcc -I"/opt/R/4.0.0/include" -DNDEBUG  -I'/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/include' -I/usr/local/include   -fpic  -g -O2  -c wrap.c -o wrap.o
Rscript execution error: Not a directory
Rscript execution error: Not a directory
g++ -std=gnu++11 -shared -L/usr/local/lib -o rhdf5.so H5.o H5A.o H5D.o H5E.o H5F.o H5G.o H5I.o H5L.o H5O.o H5P.o H5S.o H5T.o H5Z.o H5constants.o HandleList.o HandleListWrap.o h5dump.o h5ls.o h5testLock.o h5writeDataFrame.o printdatatype.o utils.o wrap.o
installing to /home/descostes/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-rhdf5/00new/rhdf5/libs
** R
** inst
** byte-compile and prepare package for lazy loading
** help
*** installing help indices
** building package indices
** installing vignettes
** testing if installed package can be loaded from temporary location
Error: package or namespace load failed for ‘rhdf5’ in dyn.load(file, DLLpath = DLLpath, ...):
 unable to load shared object '/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-rhdf5/00new/rhdf5/libs/rhdf5.so':
  /home/descostes/R/x86_64-pc-linux-gnu-library/4.0/00LOCK-rhdf5/00new/rhdf5/libs/rhdf5.so: undefined symbol: H5T_NATIVE_INT32_g
Error: loading failed
Execution halted
ERROR: loading failed
* removing ‘/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/rhdf5’

The downloaded source packages are in
    ‘/tmp/RtmpO4s2Hg/downloaded_packages’
Installation path not writeable, unable to update packages: boot, class,
  foreign, KernSmooth, MASS, mgcv, nlme, nnet, spatial, survival
Warning message:
In install.packages(...) :
  installation of package ‘rhdf5’ had non-zero exit status

Here is my session info:

> sessionInfo()
R version 4.0.0 (2020-04-24)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 18.04.4 LTS

Matrix products: default
BLAS:   /opt/R/4.0.0/lib/libRblas.so
LAPACK: /opt/R/4.0.0/lib/libRlapack.so

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C              
 [3] LC_TIME=it_IT.UTF-8        LC_COLLATE=en_US.UTF-8    
 [5] LC_MONETARY=it_IT.UTF-8    LC_MESSAGES=en_US.UTF-8   
 [7] LC_PAPER=it_IT.UTF-8       LC_NAME=C                 
 [9] LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=it_IT.UTF-8 LC_IDENTIFICATION=C       

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

loaded via a namespace (and not attached):
[1] BiocManager_1.30.10 compiler_4.0.0      tools_4.0.0

Thank you for your help.

Nicolas

[grimbough]

It looks like the linking against Rhdf5lib fails. In the the g++ -std=gnu++11 ... line there should be references to .../Rhdf5lib/lib/libhdf5.a which is where that symbol is defined.

I guess this is really down to whatever is causing the throwing of the Rscript execution error: Not a directory error.

What's the output if you run Rhdf5lib::pkgconfig("PKG_C_LIBS") and packageVersion("Rhdf5lib")?

Also, does running Rscript -e 'Rhdf5lib::pkgconfig("PKG_C_LIBS")' in a terminal throw an error?

[descostesn]

I get:

> Rhdf5lib::pkgconfig("PKG_C_LIBS")
"/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/lib/libhdf5.a"  "/home/descostes/R/x86_64-pc-linux-gnu-library/4.0/Rhdf5lib/lib/libsz.a" -lz> 
>  packageVersion("Rhdf5lib")
[1] ‘1.10.1’

and

$ Rscript -e 'Rhdf5lib::pkgconfig("PKG_C_LIBS")'
Rscript execution error: Not a directory

[grimbough]

So something is wrong with your Rscript command. That should produce the same output as when you run the command in the R session.

Reading around (e.g. https://stackoverflow.com/questions/39832110/rscript-execution-error-no-such-file-or-directory) suggests maybe R was compiled/installed in one place, and then moved? Does that sound possible?

It sounds like Rscript has a hardcoded path to R that is set at installation time, and moving might break that relationship. Maybe running which Rscript and which R would also offer some insight?

[descostesn]

I did a whole fresh install of R and still get the problem. This suggests that Rscript is not the problem to me:

descostes@PC-BIOINF-Nico:~$ Rscript
Usage: /path/to/Rscript [--options] [-e expr [-e expr2 ...] | file] [args]

--options accepted are
  --help              Print usage and exit
  --version           Print version and exit
  --verbose           Print information on progress
  --default-packages=list
                      Where 'list' is a comma-separated set
                        of package names, or 'NULL'
or options to R, in addition to --no-echo --no-restore, such as
  --save              Do save workspace at the end of the session
  --no-environ        Don't read the site and user environment files
  --no-site-file      Don't read the site-wide Rprofile
  --no-init-file      Don't read the user R profile
  --restore           Do restore previously saved objects at startup
  --vanilla           Combine --no-save, --no-restore, --no-site-file
                        --no-init-file and --no-environ

'file' may contain spaces but not shell metacharacters
Expressions (one or more '-e <expr>') may be used *instead* of 'file'
See also  ?Rscript  from within R

However I still get:

$ Rscript -e 'Rhdf5lib::pkgconfig("PKG_C_LIBS")'
Rscript execution error: No such file or directory

which looks fine to me:

descostes@PC-BIOINF-Nico:~$ which R
/usr/local/bin/R
descostes@PC-BIOINF-Nico:~$ which Rscript
/usr/local/bin/Rscript

/usr/local/bin is in my PATH.

[descostesn]

Reading more, it is indeed coming from Rscript (that print the manual even if there is a problem). For instance:

$Rscript -e 'rnorm(1)'
Rscript execution error: No such file or directory

Most of forums suggests to re-install R but I did it. If I find a solution I will put it here.

[descostesn]

So I fixed the bug. My problem was that I had multiple versions of R, lots of conflicts, etc. For some reason, installing R from the source binary did not help so I went for the Ubuntu install.

I first cleaned everything doing:

sudo apt remove rbase
sudo apt remove rbase-dev

I then deleted every executable 'R' and 'Rscript" that I could find. I found everything in /usr/ but it could have been somewhere else.

cd
cd ../..
sudo find * -name R | xargs -I{} sudo rm -r {}
sudo find * -name Rscript | xargs -I{} sudo rm -r {}

In /etc/apt/sources.list I added the line deb https://cloud.r-project.org/bin/linux/ubuntu bionic-cran40/ that I found on this page. This line may differ according to your version of Ubuntu. Then do:

sudo apt-get update
apt-cache show r-base

I selected the last version and did:

sudo apt-get install r-base=4.0.3-1.1804.0
sudo apt-get install r-base-dev=4.0.3-1.1804.0

I then changed the RHOME variable in the .bashrc:

~$ which R
/usr/bin/R
# Added the line below in the .bashrc. Note that bin/R is automatically added to the variable.
RHOME="/usr/"

I was then able to install the package. What helped me was to use the --verbose option below to realize that Rscript was not pointing to the right location:

Rscript -e 'rnorm(1)' --verbose

[grimbough]

Great to you know that fixed this, and thanks for the detailed description of how you solved it.