Support runtime compilation

[evansd]

This allows users to optimise hot loops by rewriting them in C++ and using the well-known Rcpp package.

For some reason this required upgrading ggh4x from v0.2.1 to v0.2.6 as the previous version failed to compile.

This increases the total on-disk size of the image from 2549M to 2990M.

Closes #149

[evansd]

This is currently blocked on re-compiling the ggh4x package which fails with the error:

6738.6 Error: Error installing package 'ggh4x':
6738.6 =================================
6738.6 
6738.6 * installing *source* package ‘ggh4x’ ...
6738.6 ** package ‘ggh4x’ successfully unpacked and MD5 sums checked
6738.6 ** using staged installation
6738.6 ** R
6738.6 ** inst
6738.6 ** byte-compile and prepare package for lazy loading
6738.6 Error in get(x, envir = ns, inherits = FALSE) : 
6738.6   object 'continuous_range' not found
6738.6 Error: unable to load R code in package ‘ggh4x’
6738.6 Execution halted
6738.6 ERROR: lazy loading failed for package ‘ggh4x’
6738.6 * removing ‘/renv/renv/staging/1/ggh4x’
6738.6 install of package 'ggh4x' failed [error code 1]
6738.7 Traceback (most recent calls last):
6738.7 13: renv::restore()
6738.7 12: renv_restore_run_actions(project, diff, current, lockfile, rebuild)
6738.7 11: renv_install_impl(records)
6738.7 10: renv_install_staged(records)
6738.7  9: renv_install_default(records)
6738.7  8: handler(package, renv_install_package(record))
6738.7  7: renv_install_package(record)
6738.7  6: withCallingHandlers(renv_install_package_impl(record), error = function(e) writef("FAILED"))
6738.7  5: renv_install_package_impl(record)
6738.7  4: r_cmd_install(package, path)
6738.7  3: r_exec_error(package, output, "install", status)
6738.7  2: abort(all)
6738.7  1: stop(fallback)
6738.7 Execution halted
------
Dockerfile:45
--------------------
  43 |     # use renv to install packages
  44 |     COPY renv.lock /renv/renv.lock
  45 | >>> RUN --mount=type=cache,target=/cache,id=/cache-2004 R -e 'renv::restore()'
  46 |     
  47 |     # renv uses symlinks to the the build cache to populate the lib directory. As
--------------------
ERROR: failed to solve: process "/bin/sh -c R -e 'renv::restore()'" did not complete successfully: exit code: 1
ERROR: Service 'r' failed to build : Build failed
error: Recipe `build` failed with exit code 1

If I remove the package from renv.lock then the image builds successfully: https://github.com/opensafely-core/r-docker/blob/30fc019c90ccd2934484688e93f333fb6fdbebba/renv.lock#L2048-L2061

[evansd]

As background to the above, I first tried replacing the r-base-core package in dependencies.txt with r-base-dev on the assumption that the latter pulled in the former.

When I then tried to rebuild the image I got an error building the Matrix package:

dgeMatrix.h:43:15: error: conflicting types for ‘dgesdd_’

full_log_file.txt

``` 34.2 - Installing Matrix ... FAILED 338.1 Error: Error installing package 'Matrix': 338.1 ================================== 338.1 338.1 * installing *source* package ‘Matrix’ ... 338.1 ** package ‘Matrix’ successfully unpacked and MD5 sums checked 338.1 ** using staged installation 338.1 Cleaning up after installing the Matrix package 338.1 ( cd Lib ; make clean ) 338.1 make[1]: Entering directory '/tmp/RtmpFBvrWM/R.INSTALL1b4177f25444/Matrix/src/CHOLMOD/Lib' 338.1 make[1]: Leaving directory '/tmp/RtmpFBvrWM/R.INSTALL1b4177f25444/Matrix/src/CHOLMOD/Lib' 338.1 ( cd Source ; make clean ) 338.1 make[1]: Entering directory '/tmp/RtmpFBvrWM/R.INSTALL1b4177f25444/Matrix/src/COLAMD/Source' 338.1 make[1]: Leaving directory '/tmp/RtmpFBvrWM/R.INSTALL1b4177f25444/Matrix/src/COLAMD/Source' 338.1 ( cd Source ; make clean ) 338.1 make[1]: Entering directory '/tmp/RtmpFBvrWM/R.INSTALL1b4177f25444/Matrix/src/AMD/Source' 338.1 make[1]: Leaving directory '/tmp/RtmpFBvrWM/R.INSTALL1b4177f25444/Matrix/src/AMD/Source' 338.1 ** libs 338.1 using C compiler: ‘gcc (Ubuntu 9.4.0-1ubuntu1~20.04.2) 9.4.0’ 338.1 gcc -I"/usr/share/R/include" -DNDEBUG -DNTIMER -I./SuiteSparse_config -fpic -g -O2 -fdebug-prefix-map=/build/r-base-aeZVAy/r-base-4.3.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -c CHMfactor.c -o CHMfactor.o 338.1 gcc -I"/usr/share/R/include" -DNDEBUG -DNTIMER -I./SuiteSparse_config -fpic -g -O2 -fdebug-prefix-map=/build/r-base-aeZVAy/r-base-4.3.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -c Csparse.c -o Csparse.o 338.1 gcc -I"/usr/share/R/include" -DNDEBUG -DNTIMER -I./SuiteSparse_config -fpic -g -O2 -fdebug-prefix-map=/build/r-base-aeZVAy/r-base-4.3.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -c TMatrix_as.c -o TMatrix_as.o 338.1 gcc -I"/usr/share/R/include" -DNDEBUG -DNTIMER -I./SuiteSparse_config -fpic -g -O2 -fdebug-prefix-map=/build/r-base-aeZVAy/r-base-4.3.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -c Tsparse.c -o Tsparse.o 338.1 gcc -I"/usr/share/R/include" -DNDEBUG -DNTIMER -I./SuiteSparse_config -fpic -g -O2 -fdebug-prefix-map=/build/r-base-aeZVAy/r-base-4.3.1=. -fstack-protector-strong -Wformat -Werror=format-security -Wdate-time -D_FORTIFY_SOURCE=2 -c init.c -o init.o 338.1 In file included from /usr/share/R/include/R.h:78, 338.1 from Mutils.h:12, 338.1 from init.c:1: 338.1 dgeMatrix.h:43:15: error: conflicting types for ‘dgesdd_’ 338.1 43 | void F77_NAME(dgesdd)(const char *jobz, 338.1 | ^~~~~~ 338.1 /usr/share/R/include/R_ext/RS.h:77:22: note: in definition of macro ‘F77_CALL’ 338.1 77 | # define F77_CALL(x) x ## _ 338.1 | ^ 338.1 dgeMatrix.h:43:6: note: in expansion of macro ‘F77_NAME’ 338.1 43 | void F77_NAME(dgesdd)(const char *jobz, 338.1 | ^~~~~~~~ 338.1 /usr/share/R/include/R_ext/Lapack.h:2404:10: note: previous declaration of ‘dgesdd_’ was here 338.1 2404 | F77_NAME(dgesdd)(const char* jobz, 338.1 | ^~~~~~ 338.1 /usr/share/R/include/R_ext/RS.h:77:22: note: in definition of macro ‘F77_CALL’ 338.1 77 | # define F77_CALL(x) x ## _ 338.1 | ^ 338.1 /usr/share/R/include/R_ext/Lapack.h:2404:1: note: in expansion of macro ‘F77_NAME’ 338.1 2404 | F77_NAME(dgesdd)(const char* jobz, 338.1 | ^~~~~~~~ 338.1 make: *** [/usr/lib/R/etc/Makeconf:191: init.o] Error 1 338.1 ERROR: compilation failed for package ‘Matrix’ 338.1 * removing ‘/renv/renv/staging/1/Matrix’ 338.1 install of package 'Matrix' failed [error code 1] 338.1 Traceback (most recent calls last): 338.1 13: renv::restore() 338.1 12: renv_restore_run_actions(project, diff, current, lockfile, rebuild) 338.1 11: renv_install_impl(records) 338.1 10: renv_install_staged(records) 338.1 9: renv_install_default(records) 338.1 8: handler(package, renv_install_package(record)) 338.1 7: renv_install_package(record) 338.1 6: withCallingHandlers(renv_install_package_impl(record), error = function(e) writef("FAILED")) 338.1 5: renv_install_package_impl(record) 338.1 4: r_cmd_install(package, path) 338.1 3: r_exec_error(package, output, "install", status) 338.1 2: abort(all) 338.1 1: stop(fallback) 338.1 Execution halted ------ Dockerfile:45 -------------------- 43 | # use renv to install packages 44 | COPY renv.lock /renv/renv.lock 45 | >>> RUN --mount=type=cache,target=/cache,id=/cache-2004 R -e 'renv::restore()' 46 | 47 | # renv uses symlinks to the the build cache to populate the lib directory. As -------------------- ERROR: failed to solve: process "/bin/sh -c R -e 'renv::restore()'" did not complete successfully: exit code: 1 ERROR: Service 'r' failed to build : Build failed error: Recipe `build` failed with exit code 1 ```

I thought maybe this was some problem with the BuildKit cache and so I wiped that using:

docker builder prune --filter type=exec.cachemount

And then waited the 45 years necessary to recompile everything from scratch, but that still gave the same error.

After changing dependencies.txt to include both r-base-dev and r-base-core the Matrix package compiled successfully, but then I hit the issue with ggh4x. After removing that, the image built successfully.