search

hhoeflin / hdf5r

Other
80 stars 23 forks source link

Run ASAN, valgrind tests for package #54

Closed mannau closed 7 years ago

mannau commented 7 years ago

Copied from the h5 Makefile:

check-asan-gcc: $(PKG_NAME)_$(PKG_VERSION).tar.gz
    @boot2docker up
    $(shell boot2docker shellinit)
    @docker run -v "$(CURRENT_DIR):/mnt" mannau/r-devel-san /bin/bash -c \
        "cd /mnt; apt-get update; apt-get clean; apt-get install -y libhdf5-dev; \
        R -e \"install.packages(c('Rcpp', 'testthat', 'roxygen2', 'highlight', 'zoo', 'microbenchmark'))\"; \
        R CMD check $(PKG_NAME)_$(PKG_VERSION).tar.gz; \
        cat /mnt/h5.Rcheck/00install.out"

check-valgrind: $(PKG_NAME)_$(PKG_VERSION).tar.gz 
    @rm -rf $(CHECKPATH)
    $(R) CMD check --no-clean --use-valgrind $(PKG_NAME)_$(PKG_VERSION).tar.gz
mannau commented 7 years ago

R CMD check --use-valgrind looks good on https://github.com/hhoeflin/hdf5r/commit/f71d57558c305af2b40993114620371a489fc448 using OSX 10.12.6, Apple LLVM version 8.1.0 (clang-802.0.42), valgrind-3.13.0:


==> devtools::check(document = FALSE, args = c('--no-clean','--use-valgrind'))

Setting env vars --------------------------------------------------------------
CFLAGS  : -Wall -pedantic
CXXFLAGS: -Wall -pedantic
Building hdf5r ----------------------------------------------------------------
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ  \
  --no-save --no-restore --quiet CMD build  \
  '/Users/mario/CloudStation/Programming/hdf5/hdf5r' --no-resave-data  \
  --no-manual 

* checking for file ‘/Users/mario/CloudStation/Programming/hdf5/hdf5r/DESCRIPTION’ ... OK
* preparing ‘hdf5r’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* installing the package to build vignettes
* creating vignettes ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files
* checking for empty or unneeded directories
* building ‘hdf5r_0.9.7.9000.tar.gz’

Setting env vars --------------------------------------------------------------
_R_CHECK_CRAN_INCOMING_ : FALSE
_R_CHECK_FORCE_SUGGESTS_: FALSE
Checking hdf5r ----------------------------------------------------------------
'/Library/Frameworks/R.framework/Resources/bin/R' --no-site-file --no-environ  \
  --no-save --no-restore --quiet CMD check  \
  '/var/folders/q3/0dypvmy96jv26m3qk49pp85r0000gn/T//RtmpdIc7uw/hdf5r_0.9.7.9000.tar.gz'  \
  --as-cran --timings --no-clean --use-valgrind --no-manual 

* using log directory ‘/Users/mario/CloudStation/Programming/hdf5/hdf5r.Rcheck’
* using R version 3.4.0 (2017-04-21)
* using platform: x86_64-apple-darwin15.6.0 (64-bit)
* using session charset: UTF-8
* using options ‘--no-manual --use-valgrind --as-cran’
* checking for file ‘hdf5r/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘hdf5r’ version ‘0.9.7.9000’
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘hdf5r’ can be installed ... OK
* checking installed package size ... OK
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd line widths ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... OK
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking compiled code ... OK
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testthat.R’ OK
* checking for unstated dependencies in vignettes ... [12m/12m]
 OK
* checking package vignettes in ‘inst/doc’ ... OK
* checking re-building of vignette outputs ... OK
* DONE
Status: OK

R CMD check results
0 errors | 0 warnings | 0 notes

R CMD check succeeded
mannau commented 7 years ago

Checking with Clang4.0 and Adress sanitizers also looks good now (thanks to https://github.com/hhoeflin/hdf5r/commit/4de064508d3850ac5d3e8e7da214b443855d63bb):

Marios-MacBook-Pro-2:hdf5r mario$ R CMD build .;docker run --cap-add SYS_PTRACE -v "$(pwd):/mnt" f023928cf1d8 /bin/bash -c "cd /mnt; Rdevel CMD check *.tar.gz;cat /mnt/hdf5r.Rcheck/00install.out"
* checking for file ‘./DESCRIPTION’ ... OK
* preparing ‘hdf5r’:
* checking DESCRIPTION meta-information ... OK
* cleaning src
* running ‘cleanup’
* installing the package to build vignettes
* creating vignettes ... OK
* cleaning src
* running ‘cleanup’
* checking for LF line-endings in source and make files
* checking for empty or unneeded directories
* building ‘hdf5r_0.9.7.9000.tar.gz’

* using log directory ‘/mnt/hdf5r.Rcheck’
* using R Under development (unstable) (2017-08-29 r73159)
* using platform: x86_64-pc-linux-gnu (64-bit)
* using session charset: UTF-8
* checking for file ‘hdf5r/DESCRIPTION’ ... OK
* checking extension type ... Package
* this is package ‘hdf5r’ version ‘0.9.7.9000’
* checking package namespace information ... OK
* checking package dependencies ... OK
* checking if this is a source package ... OK
* checking if there is a namespace ... OK
* checking for executable files ... OK
* checking for hidden files and directories ... OK
* checking for portable file names ... OK
* checking for sufficient/correct file permissions ... OK
* checking whether package ‘hdf5r’ can be installed ... OK
* checking installed package size ... NOTE
  installed size is  5.0Mb
  sub-directories of 1Mb or more:
    libs   2.1Mb
* checking package directory ... OK
* checking ‘build’ directory ... OK
* checking DESCRIPTION meta-information ... OK
* checking top-level files ... OK
* checking for left-over files ... OK
* checking index information ... OK
* checking package subdirectories ... OK
* checking R files for non-ASCII characters ... OK
* checking R files for syntax errors ... OK
* checking whether the package can be loaded ... OK
* checking whether the package can be loaded with stated dependencies ... OK
* checking whether the package can be unloaded cleanly ... OK
* checking whether the namespace can be loaded with stated dependencies ... OK
* checking whether the namespace can be unloaded cleanly ... OK
* checking loading without being on the library search path ... OK
* checking dependencies in R code ... OK
* checking S3 generic/method consistency ... OK
* checking replacement functions ... OK
* checking foreign function calls ... OK
* checking R code for possible problems ... OK
* checking Rd files ... OK
* checking Rd metadata ... OK
* checking Rd cross-references ... OK
* checking for missing documentation entries ... OK
* checking for code/documentation mismatches ... OK
* checking Rd \usage sections ... OK
* checking Rd contents ... OK
* checking for unstated dependencies in examples ... OK
* checking line endings in shell scripts ... OK
* checking line endings in C/C++/Fortran sources/headers ... OK
* checking line endings in Makefiles ... OK
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... OK
* checking for portable use of $(BLAS_LIBS) and $(LAPACK_LIBS) ... OK
* checking compiled code ... OK
* checking installed files from ‘inst/doc’ ... OK
* checking files in ‘vignettes’ ... OK
* checking examples ... OK
* checking for unstated dependencies in ‘tests’ ... OK
* checking tests ...
  Running ‘testthat.R’
 OK
* checking for unstated dependencies in vignettes ... OK
* checking package vignettes in ‘inst/doc’ ... OK
* checking running R code from vignettes ...
   ‘hdf5r.Rmd’ using ‘UTF-8’ ... OK
 NONE
* checking re-building of vignette outputs ... OK
* checking PDF version of manual ... OK
* DONE

Status: 1 NOTE
See
  ‘/mnt/hdf5r.Rcheck/00check.log’
for details.

00install.out.txt

The basis for the docker container used is available at https://github.com/mannau/r-devel-san-clang/blob/master/Dockerfile. I would like to use it as a wercker build in the future.

hhoeflin commented 7 years ago

Excellent - good to see that valgrind is happy :)

Can you explain a little more about "wercker"?

mannau commented 7 years ago

Wercker is a CI integration tool (like travis and appveyor) which allows to run builds in custom docker containers. I just added a wercker application to check the package using Adress Sanitizers and valgrind. For ASAN and valgrind tests I have developed and pushed 2 docker images on dockerhub:

These images are based on Dockerfile github repos and are rebuilt after each git push or if the docker images they are based on are changed.

mannau commented 7 years ago

Valgrind CI tests have now been disabled on Wercker due to timeouts. ASAN tests work as expected.