search

grimbough / rhdf5

Package providing an interface between HDF5 and R
http://bioconductor.org/packages/rhdf5
59 stars 22 forks source link

H5Dset_extent failing on ARM64 #123

Closed grimbough closed 1 year ago

grimbough commented 1 year ago

Previously reported in https://github.com/ArtifactDB/alabaster.matrix/issues/2

grimbough commented 1 year ago

@martin-g I added some additional testing for H5Dset_extent() in https://github.com/grimbough/rhdf5/commit/accb7c4d8c4e08a17b8cf16a271ccf9446d53d98 It should be available from both Github and the Bioc devel branch. Let me know if it sheds any more light on the alabaster.matrix problem.

martin-g commented 1 year ago

Thanks! I will re-test after lunch!

martin-g commented 1 year ago

Here are the results for latest devel:

biocbuild@kunpeng1 ~/git> R CMD build --keep-empty-dirs --no-resave-data rhdf5
* checking for file ‘rhdf5/DESCRIPTION’ ... OK
* preparing ‘rhdf5’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* installing the package to build vignettes
* creating vignettes ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘rhdf5_2.43.2.tar.gz’
biocbuild@kunpeng1 ~/git> R CMD check rhdf5_2.43.2.tar.gz 
* using log directory ‘/home/biocbuild/git/rhdf5.Rcheck’
* using R Under development (unstable) (2023-03-12 r83975)
* using platform: aarch64-unknown-linux-gnu (64-bit)
* R was compiled by
    gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
    GNU Fortran (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
* running under: Ubuntu 22.04.2 LTS
* using session charset: UTF-8
* checking for file ‘rhdf5/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘rhdf5’ version ‘2.43.2’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘rhdf5’ can be installed ... OK
* used C compiler: ‘gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0’
* used C++ compiler: ‘g++ (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0’
* checking installed package size ... NOTE
  installed size is 16.6Mb
  sub-directories of 1Mb or more:
    doc    2.3Mb
    libs  13.4Mb
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking startup messages can be suppressed ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... WARNING
Undocumented arguments in documentation object 'H5Dwrite'
  ‘h5type’

Functions with \usage entries need to have the appropriate \alias
entries, and all their arguments documented.
The \usage entries must correspond to syntactically valid R code.
See chapter ‘Writing R documentation files’ in the ‘Writing R
Extensions’ manual.
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking line endings in shell scripts ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... NOTE
GNU make is a SystemRequirements.
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking use of PKG_*FLAGS in Makefiles ... OK
* checking compiled code ... WARNING
File ‘rhdf5/libs/rhdf5.so’:
  Found ‘__sprintf_chk’, possibly from ‘sprintf’ (C)
    Objects: ‘H5A.o’, ‘H5D.o’, ‘H5E.o’, ‘wrap.o’

Compiled code should not call entry points which might terminate R nor
write to stdout/stderr instead of to the console, nor use Fortran I/O
nor system RNGs nor [v]sprintf.

See ‘Writing portable packages’ in the ‘Writing R Extensions’ manual.
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testthat.R’
 ERROR
Running the tests in ‘tests/testthat.R’ failed.
Last 13 lines of output:
      ▆
   1. └─rhdf5::h5createDataset(...) at test_h5create.R:179:4
   2.   └─rhdf5::H5Dcreate(loc$H5Identifier, dataset, tid, sid, dcpl = dcpl)
   3.     └─rhdf5:::h5checktype(h5space, "dataspace")
  ── Error ('test_h5create.R:179:5'): Extendible datasets ────────────────────────
  Error: Error in h5checktype(). Argument not of class H5IdComponent.
  Backtrace:
      ▆
   1. └─rhdf5::h5createDataset(...) at test_h5create.R:179:4
   2.   └─rhdf5::H5Sclose(sid) at test_h5create.R:179:4
   3.     └─rhdf5:::h5checktype(h5space, "dataspace")

  [ FAIL 33 | WARN 0 | SKIP 2 | PASS 1021 ]
  Error: Test failures
  Execution halted
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ‘inst/doc’ ... OK
* checking running R code from vignettes ...
  ‘practical_tips.Rmd’ using ‘UTF-8’... OK
  ‘rhdf5.Rmd’ using ‘UTF-8’... OK
  ‘rhdf5_cloud_reading.Rmd’ using ‘UTF-8’... OK
 NONE
* checking re-building of vignette outputs ...
 OK
* checking PDF version of manual ... OK
* DONE

Status: 1 ERROR, 2 WARNINGs, 2 NOTEs
See
  ‘/home/biocbuild/git/rhdf5.Rcheck/00check.log’
for details.

The failure seems unrelated to the issue in alabaster.matrix. I will test now with the previous commit (the one before https://github.com/grimbough/rhdf5/commit/accb7c4d8c4e08a17b8cf16a271ccf9446d53d98).

grimbough commented 1 year ago

I think this was due to the way the HDF5 constant H5S_UNLIMITED was being used. It's an unsigned 64-bit int in the HDF5 library and I think the way it was being based between C and R was failing for some reason on ARM64 but not AMD64.

I've switched to a simpler and more explicit strategy, and now all the tests are passing on my example workflow (https://github.com/grimbough/rhdf5/actions/runs/4612713155/jobs/8153967201).

Would be great if you could test the latest version on your system when you get a chance.

martin-g commented 1 year ago

All looks good with the latest version! Good work, @grimbough !

biocbuild@kunpeng1 ~/git> R CMD build --keep-empty-dirs --no-resave-data rhdf5
* checking for file ‘rhdf5/DESCRIPTION’ ... OK
* preparing ‘rhdf5’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* installing the package to build vignettes
* creating vignettes ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files and shell scripts
* checking for empty or unneeded directories
* building ‘rhdf5_2.43.3.tar.gz’
biocbuild@kunpeng1 ~/git> R CMD check rhdf5_2.43.3.tar.gz 
* using log directory ‘/home/biocbuild/git/rhdf5.Rcheck’
* using R Under development (unstable) (2023-03-12 r83975)
* using platform: aarch64-unknown-linux-gnu (64-bit)
* R was compiled by
    gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
    GNU Fortran (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0
* running under: Ubuntu 22.04.2 LTS
* using session charset: UTF-8
* checking for file ‘rhdf5/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘rhdf5’ version ‘2.43.3’
* package encoding: UTF-8
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘rhdf5’ can be installed ... OK
* used C compiler: ‘gcc (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0’
* used C++ compiler: ‘g++ (Ubuntu 11.3.0-1ubuntu1~22.04) 11.3.0’
* checking installed package size ... NOTE
  installed size is 16.6Mb
  sub-directories of 1Mb or more:
    doc    2.3Mb
    libs  13.4Mb
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking startup messages can be suppressed ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking line endings in shell scripts ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... NOTE
GNU make is a SystemRequirements.
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking use of PKG_*FLAGS in Makefiles ... OK
* checking compiled code ... OK
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testthat.R’
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ‘inst/doc’ ... OK
* checking running R code from vignettes ...
  ‘practical_tips.Rmd’ using ‘UTF-8’... OK
  ‘rhdf5.Rmd’ using ‘UTF-8’... OK
  ‘rhdf5_cloud_reading.Rmd’ using ‘UTF-8’... OK
 NONE
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* DONE

Status: 2 NOTEs
See
  ‘/home/biocbuild/git/rhdf5.Rcheck/00check.log’
for details.
grimbough commented 1 year ago

Awesome, thanks for testing it.