extendr / rextendr

An R package that helps scaffolding extendr-enabled packages or compiling Rust code dynamically
https://extendr.github.io/rextendr/
Other
181 stars 27 forks source link

R CMD check prevents caching and raise warnings on cargo build cache. #232

Open sorhawell opened 1 year ago

sorhawell commented 1 year ago

check prevents caching Whenever performing R CMD check or a wrapper thereof as devtools::check(), rcmdcheck or r-lib/actions/check-r-package on a rextendr-powered package, then a tempoary folder will be created moving only source files, not a potentially previous cargo build. The entire cargo package will be compiled from scratch.

typical check dirs are (replace r-polars/r-polars with any project/package name) Windows: D:/a/r-polars/r-polars/check/rpolars.Rcheck/ Linux /home/runner/work/r-polars/r-polars/check/rpolars.Rcheck/ MacOS /Users/runner/work/r-polars/r-polars/check/rpolars.Rcheck/

check will always fail(raise warning) on certain crates When running cargo build inside the "temp-R-CMD-check-folder" R CMD check will with little meaningfulness check build scripts of any crate dependency. Such crates can adhere to completely different standards and emailing several crate maintainer and convincing them to adhere to R CMD check will likely fail or at least be flaky and not very fun. So far I have seen check warnings on: Missing line endings:

Warning: Found the following directories with names of version control directories:
  ./src/.cargo/git/checkouts/arrow2-945af624853845da/baa2618/.git
  ./src/.cargo/git/checkouts/arrow2-945af624853845da/baa2618/testing/arrow-testing/.git
  ./src/.cargo/git/checkouts/arrow2-945af624853845da/baa2618/testing/parquet-testing/.git
  ./src/.cargo/git/checkouts/extendr-569c02e1a164cdf5/f511254/.git
  ./src/.cargo/git/checkouts/jsonpath-3ee2ec25574f7416/24eaf0b/.git
  ./src/.cargo/git/checkouts/polars-b0d90607192fd414/a7d38dd/.git
  ./src/.cargo/registry/index/github.com-1ecc6299db9ec823/.git

use of modern makefiles and missing line endings:

Found the following sources/headers with CR or CRLF line endings:
  src/rust/target/x86_64-pc-windows-gnu/release/build/libz-ng-sys-244194a103d06ae8/out/build/CMakeFiles/3.24.1/CompilerIdC/CMakeCCompilerId.c
  src/rust/target/x86_64-pc-windows-gnu/release/build/libz-ng-sys-244194a103d06ae8/out/build/CMakeFiles/CheckTypeSize/OFF64_T.c
  src/rust/target/x86_64-pc-windows-gnu/release/build/libz-ng-sys-244194a103d06ae8/out/build/gzread.c
  src/rust/target/x86_64-pc-windows-gnu/release/build/libz-ng-sys-244194a103d06ae8/out/build/zconf-ng.h
  src/rust/target/x86_64-pc-windows-gnu/release/build/libz-ng-sys-244194a103d06ae8/out/build/zlib-ng.h
  src/rust/target/x86_64-pc-windows-gnu/release/build/libz-ng-sys-244194a103d06ae8/out/include/zconf-ng.h
  src/rust/target/x86_64-pc-windows-gnu/release/build/libz-ng-sys-244194a103d06ae8/out/include/zlib-ng.h
Some Unix compilers require LF line endings.
* checking line endings in Makefiles ... WARNING
Warning: Found the following Makefile(s) with CR or CRLF line endings:
  src/rust/target/x86_64-pc-windows-gnu/release/build/libz-ng-sys-244194a103d06ae8/out/build/Makefile
Some Unix 'make' programs require LF line endings.
* checking compilation flags in Makevars ... OK
* checking for GNU extensions in Makefiles ... WARNING
Warning: Found the following file(s) containing GNU extensions:
  src/rust/target/x86_64-pc-windows-gnu/release/build/libz-ng-sys-244194a103d06ae8/out/build/Makefile
Portable Makefiles do not use GNU extensions such as +=, :=, $(shell),
$(wildcard), ifeq ... endif, .NOTPARALLEL See section 'Writing portable
packages' in the 'Writing R Extensions' manual.

certain git files, in combination with CARGO_HOME/CRAN fix

 Warning: Found the following directories with names of version control directories:
  ./src/.cargo/git/checkouts/arrow2-945af624853845da/baa2618/.git
  ./src/.cargo/git/checkouts/arrow2-945af624853845da/baa2618/testing/arrow-testing/.git
  ./src/.cargo/git/checkouts/arrow2-945af624853845da/baa2618/testing/parquet-testing/.git
  ./src/.cargo/git/checkouts/extendr-569c02e1a164cdf5/f511254/.git
  ./src/.cargo/git/checkouts/jsonpath-3ee2ec25574f7416/24eaf0b/.git
  ./src/.cargo/git/checkouts/polars-b0d90607192fd414/a7d38dd/.git
  ./src/.cargo/registry/index/github.com-1ecc6299db9ec823/.git

One quick fix is to delete all the cargo build files except produces binaries. That would however prevent caching.

A better fix is to not cargo build in "temp-R-CMD-check-folder" and only symlink the final binary into the folder. Then caching can proceed as normal and no check-illegal files will be created there. One example solution can be found here and run can look like this, see print "print check install - log"

This solution does yet not include CARGO_HOME which I think just need a bit more spring cleaning to avoid the illegal git files in .cargo/

I will try to write a PR when I have tested out.