Adding Apptainer/Singularity support

[e-kotov]

I have added support for Apptainer (formerly Singularity) containers. To do this I:

• moved a few helper functions from dockerize.R to installarion.R, specifically: .insert_materials_dir() and .write_dockerfile(). Also .write_dockerfile() is now more generic .write_container_file(), because it serves both dockerize() and apptainerize(). .insert_materials_dir() now also serves both functions and distinguishes between docker and apptainer/singularity containers. It seems safe to do so, all existing tests pass and I also added a separate test for it in test_apptainerize.R.
• added apptainerize() function (with name variations). I am not sure you want to "pollute" the package with singularize() synonyms. Kindly let me know what you think.
• covered apptainerize() with formal tests that mirror what you have for `dockerize()' + I did a lot of testing using the actual singularity module on a linux server. All singularity definitions seem to work just fine.
• caching of debian rootfs tar does not work in an easy way for Apptainer/Singularity, as it requires user to have debootstrap installed https://apptainer.org/docs/user/latest/appendix.html#id17 therefore, at least for now, I think it is much more user friendly to skip the caching of Debian image and just use the Docker image of debian:eol in the Apptainer/Singularity definition.
• updated README and FAQ vignette
• extensively tested using Linux VPS, all Apptainer/Singularity image definition files that I get out of the new function seem to build without issues.

[codecov-commenter]

Codecov Report

Merging #134 (b546f78) into v0.3 (d665fdf) will decrease coverage by 1.77%. The diff coverage is 84.93%.

:mega: This organization is not using Codecov’s GitHub App Integration. We recommend you install it so Codecov can continue to function properly for your repositories. Learn more

@@            Coverage Diff             @@
##             v0.3     #134      +/-   ##
==========================================
- Coverage   97.26%   95.50%   -1.77%     
==========================================
  Files          10       11       +1     
  Lines         988     1157     +169     
==========================================
+ Hits          961     1105     +144     
- Misses         27       52      +25     

Impacted Files	Coverage Δ
R/dockerfile.R	100.00% <ø> (ø)
R/apptainer.R	82.53% <82.53%> (ø)
R/installation.R	93.97% <86.40%> (-3.45%)	:arrow_down:

... and 1 file with indirect coverage changes

:mega: We’re building smart automated test selection to slash your CI/CD build times. Learn more

[chainsawriot]

Smoke

• [x] resolve("github::cran/psych", snapshot_date = "2013-01-01") (if prepend fixed)
• [x] resolve("chimeraviz", snapshot_date = "2023-01-01")
• [x] resolve("chimeraviz", snapshot_date = "2023-01-01") + quarto
• [x] resolve("chimeraviz", snapshot_date = "2023-01-01") + material (although the materials_dir seems to work differently to docker)
• [x] resolve(pkgs = "quanteda", snapshot_date = "2018-10-06") + rstudio

[chainsawriot]

@e-kotov Did you try to apptainerize with image = "rstudio"? I can't make it work.

somewhere <- ""
x <- resolve(pkgs = "quanteda", snapshot_date = "2018-10-06")
apptainerize(x, somewhere, image = "rstudio")

It can be built fine.

cd somewhere
apptainer build container.sif container.def

But can't be launched correctly. Specifically /init won't run.

 ## wouldn't run, with error
apptainer instance start container.sif
## can access via 0.0.0.0:8787, but error
apptainer exec container.sif /usr/lib/rstudio-server/bin/rserver --auth-none=1 --auth-pam-helper-path=pam-helper 
## password won't pass
PASSWORD='abc123' singularity exec container.sif /usr/lib/rstudio-server/bin/rserver --auth-none=0 --auth-pam-helper-path=pam-helper 

But I must admit I am a pretty lousy apptainer user (just learned to use it 72 hours ago). Maybe you would have a better idea how it works.

[e-kotov]

@chainsawriot I have tested, but in a slightly different environment. I am woking on (a) fixes and (b) readme for apptainer. Some more testing needs to be done, as it does not work in every environment reliably yet. I have addressed most of your comments already, but I have not pushed my commits to GitHub yet. I will let you know once I re-check everything so that you don't have to spend your time troubleshooting.

Thanks for your feedback and patience.

[e-kotov]

@e-kotov Did you try to apptainerize with image = "rstudio"? I can't make it work.

I did some more testing and here are the details on how to run those. I am also adding the info below to package readme, Apptainer readme template and FAQ vignette.

To run RStudio IDE in Apptainer/Singularity container, some writeable folders and a config file have to be created locally:

mkdir -p run var-lib-rstudio-server .rstudio
printf 'provider=sqlite\ndirectory=/var/lib/rstudio-server\n' > database.conf

After that, you can run the container (do not run as root user, otherwise you will not be able to login to RStudio IDE).

Start instance (on default RSTUDIO port 8787):

apptainer instance start \
    --bind run:/run,var-lib-rstudio-server:/var/lib/rstudio-server,database.conf:/etc/rstudio/database.conf,.rstudio:/home/rstudio/.rstudio/ \
    container.sif \
    rangtest

Now open a browser and go to localhost:8787. The default username is your local username, default password is 'set_your_password' (if you are using container generated by rang).

List running instances:

apptainer instance list

Stop instance:

apptainer instance stop rangtest

Start instance with custom port (e.g. 8080) and password:

apptainer instance start \
    --enc RPORT=8080
    --env PASSWORD='set_your_password' \
    --bind run:/run,var-lib-rstudio-server:/var/lib/rstudio-server,database.conf:/etc/rstudio/database.conf,.rstudio:/home/rstudio/.rstudio/ \
    container.sif \
    rangtest

Run container with custom rserver command line:

apptainer exec \
    --env PASSWORD='set_your_password' \
    --bind run:/run,var-lib-rstudio-server:/var/lib/rstudio-server,database.conf:/etc/rstudio/database.conf,.rstudio:/home/rstudio/.rstudio/ \
    container.sif \
    /usr/lib/rstudio-server/bin/rserver \
    --auth-none=0 --auth-pam-helper-path=pam-helper \
    --server-user=$(whoami) --www-port=8787

If you run the container using apptainer exec command, you will have to kill the rserver process manually or Cmd/Ctrl+C from the running container to stop the server.

[e-kotov]

@chainsawriot I have addressed all of your review suggestions and questions and have done some more testing on a VPS with both admin and user privileges. I hope my README additions will easily allow you to check the updates and successfully run an Apptainer container.

[e-kotov]

For quicker tests, I use:

somewhere <- ""
x <- resolve(pkgs = "data.table", snapshot_date = "2019-10-06")
apptainerize(x, somewhere, image = "rstudio")

It is not 'foolproof', but for quick iteration it is ideal, as data.table has no dependencies and builds very quickly during container builds.

[chainsawriot]

@e-kotov Thanks a lot! I can get all smoke tests running. I will merge it now. I think there are still some minor issues related to the documentation. But those can be fixed later.

Again, thanks a lot for your tremendous contribution to this package!

[e-kotov]

@chainsawriot Thank you too! I will look out for issues on your repo related to Apptainer/Singularity.