search

rstudio / renv

renv: Project environments for R.
https://rstudio.github.io/renv/
MIT License
1.02k stars 155 forks source link

Podman build not working after testing init() in container #1335

Closed IlmariTamminen closed 1 year ago

IlmariTamminen commented 1 year ago

Hi.

I was experimenting with the renv package in a rocker/r-ver:4.2.1 container run with the Podman 4.5.0. To build the image and deploy the containers, I use the podman-machine-default in Windows 10 with WSL2. I openend a bash terminal into the running rootless Rocker container, started R 4.2.1 instance there, ran install.packages("renv") and then:

> init()

Welcome to renv!

It looks like this is your first time using renv. This is a one-time message,
briefly describing some of renv's functionality.

renv maintains a local cache of data on the filesystem, located at:

  - "~/.cache/R/renv"

This path can be customized: please see the documentation in `?renv::paths`.

renv will also write to files within the active project folder, including:

  - A folder 'renv' in the project directory, and
  - A lockfile called 'renv.lock' in the project directory.

In particular, projects using renv will normally use a private, per-project
R library, in which new packages will be installed. This project library is
isolated from other R libraries on your system.

In addition, renv will update files within your project directory, including:

  - .gitignore
  - .Rbuildignore
  - .Rprofile

Please read the introduction vignette with `vignette("renv")` for more information.
You can browse the package documentation online at https://rstudio.github.io/renv/.
Do you want to proceed? [y/N]:

>y
* "~/.cache/R/renv" has been created.
* Initializing project ...
cp -pPR '/usr/local/lib/R/site-library/renv/' '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19'
================================================================================

cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/DESCRIPTION': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/INDEX': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/LICENSE': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/Meta/Rd.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/Meta/features.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/Meta/hsearch.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/Meta/links.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/Meta/nsInfo.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/Meta/package.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/Meta/vignette.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/Meta': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/NAMESPACE': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/NEWS.md': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/R/renv': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/R/renv.rdb': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/R/renv.rdx': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/bin/renv': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/bin/renv.bat': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/bin': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/config.yml': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/bioconductor.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/bioconductor.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/bioconductor.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/cellar.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/cellar.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/cellar.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/ci.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/ci.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/ci.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/collaborating.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/collaborating.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/collaborating.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/docker.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/docker.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/docker.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/faq.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/faq.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/faq.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/index.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/lockfile.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/lockfile.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/lockfile.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/mran.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/mran.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/mran.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/packages.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/packages.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/packages.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/profiles.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/profiles.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/profiles.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/python.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/python.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/python.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/renv.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/renv.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/renv.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/rsconnect.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/rsconnect.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/rsconnect.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/use.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/use.Rmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc/use.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/doc': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/help/AnIndex': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/help/aliases.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/help/figures/logo.svg': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/help/figures': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/help/paths.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/help/renv.rdb': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/help/renv.rdx': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/help': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/html/00Index.html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/html/R.css': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/html': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/repos/src/contrib/PACKAGES': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/repos/src/contrib/PACKAGES.gz': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/repos/src/contrib/PACKAGES.rds': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/repos/src/contrib/renv_0.16.0.tar.gz': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/repos/src/contrib': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/repos/src': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/repos': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/resources/WELCOME': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/resources/activate.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/resources/scripts-git-askpass.cmd': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/resources/scripts-git-askpass.sh': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/resources/scripts-repos-cache.R': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/resources': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/rstudio/addins.dcf': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19/rstudio': Operation not permitted
cp: preserving times for '/container/renv/library/R-4.2/x86_64-pc-linux-gnu/.renv-copy-1635d4dd19': 
Operation not permitted
Error: error copying directory [error code 1]
Traceback (most recent calls last):
7: init()
6: renv_imbue_impl(project)
5: renv_imbue_self(project)
4: renv_imbue_self_binary(source, target)
3: renv_file_copy(source, target, overwrite = TRUE)
2: renv_file_copy_dir(source, target)
1: stop(status)

Because the previous run was problematic, I also tried init(bare = TRUE) with similar results. Then, when I tried to build another Rocker image (which shouldn't have nothing to do with the renv) the Podman failed:

podman build -t testname/testcontainer:1.0 .
time="2023-05-24T12:18:29+03:00" level=error msg="1 error occurred:\n\t* open C:\\some_directories\\renv\\sandbox\\R-4.2\\x86_64-pc-linux-gnu\\25ebdc09\\KernSmooth: The file cannot be accessed by the system.\n\n\n"
Error: Post "http://d/v4.5.0/libpod/build?dockerfile=%5B%22Dockerfile%22%5D&forcerm=1&httpproxy=1&identitylabel=1&idmappingoptions=%7B%22HostUIDMapping%22%3Atrue%2C%22HostGIDMapping%22%3Atrue%2C%22UIDMap%22%3A%5B%5D%2C%22GIDMap%22%3A%5B%5D%2C%22AutoUserNs%22%3Afalse%2C%22AutoUserNsOpts%22%3A%7B%22Size%22%3A0%2C%22InitialSize%22%3A0%2C%22PasswdFile%22%3A%22%22%2C%22GroupFile%22%3A%22%22%2C%22AdditionalUIDMappings%22%3Anull%2C%22AdditionalGIDMappings%22%3Anull%7D%7D&isolation=1&jobs=1&layers=1&networkmode=0&nsoptions=%5B%7B%22Name%22%3A%22user%22%2C%22Host%22%3Atrue%2C%22Path%22%3A%22%22%7D%5D&omithistory=0&output=testname%2Ftestcontainer%3A1.0&outputformat=application%2Fvnd.oci.image.manifest.v1%2Bjson&platform=%2F&pullpolicy=missing&rm=1&shmsize=67108864&t=testname%2Ftestcontainer%3A1.0": 
io: read/write on closed pipe

Now I am stuck without working podman build. I believed that experimenting with the renv in the rootless Rocker container would be safe, but something obviously went wrong. As carefully as I can, I removed and reinstalled both the Podman and the WSL following these recommendations, but the issue didn't gone away: https://superuser.com/questions/1619233/completely-reinstall-wsl/1755410#1755410

So, how could I fully revert back to the state where I was before trying the renv package and make the podman build etc. work again? Furthermore, how could I use the renv package on a Linux computer, running R/4.2.1-foss-2022a module, for installing the exactly same R package versions in the Rocker 4.2.1 image build on the Windows computer? I do not want to change anything "dramatic" or update nothing on the Linux computer due these risks, just transfer the information about the installed R packages and their versions to my Rocker image on Windows while being absolutely sure that nothing is screwed on the computers. I don't have an R project on the Linux computer if it matters, I just run R scripts there.

kevinushey commented 1 year ago

Sorry, but I have no experience with podman.

time="2023-05-24T12:18:29+03:00" level=error msg="1 error occurred:\n\t* open C:\some_directories\renv\sandbox\R-4.2\x86_64-pc-linux-gnu\25ebdc09\KernSmooth: The file cannot be accessed by the system.\n\n\n"

The error these suggests you lack the requisite permissions to use the renv sandbox; you might need to manually fix those permissions, or delete that sandbox folder (as root).

IlmariTamminen commented 1 year ago

Sorry, but I have no experience with podman.

time="2023-05-24T12:18:29+03:00" level=error msg="1 error occurred:\n\t* open C:\some_directories\renv\sandbox\R-4.2\x86_64-pc-linux-gnu\25ebdc09\KernSmooth: The file cannot be accessed by the system.\n\n\n"

The error these suggests you lack the requisite permissions to use the renv sandbox; you might need to manually fix those permissions, or delete that sandbox folder (as root).

Many thanks for your help! Okay, this is a little bit complicated because I am not familiar (yet) with the renv. Some relevant background could help me with the troubleshooting. Considering the context and what has been done, what is this "sandboxing" and why this feature turned on? Is it default behaviour when the initialization is done?

It is pretty odd that the podman even knows about the existence of the renv. The virtual Linux and R instances were destroyed long time ago with the shutdown container in which the renv was tested. I do see that a renv folder appeared in the directory (mounted earlier in the now destroyed container) on the Windows where my dockerfile is, the file that determines how my image is build. But the dockerfile hasn't been changed, the Rocker image used as the base for building my image hasn't changed, and as said eventually everything were wiped out during the reinstallations. Thus, where exactly should I look for changes on the computer? Excluding the appearance of the renv folder, and possibly everything that is related to the virtual Linux environment, because those should be reset now.

kevinushey commented 1 year ago

Many thanks for your help! Okay, this is a little bit complicated because I am not familiar (yet) with the renv. Some relevant background could help me with the troubleshooting. Considering the context and what has been done, what is this "sandboxing" and why this feature turned on? Is it default behaviour when the initialization is done?

This is described in https://rstudio.github.io/renv/dev/reference/sandbox.html -- in essence, it's a pseudo-library of base / recommended R packages, and it's marked as non-writable so users don't accidentally install packages into that path when using renv.

https://rstudio.github.io/renv/dev/reference/paths.html may also be helpful for seeing where renv stores different global state.

hadley commented 1 year ago

Since it's now been two weeks, and it doesn't seem like there's anything renv needs to do, I'm going to close this issue.

IlmariTamminen commented 1 year ago

Sorry, I had my own delays and pending questions for Podman. Maybe the issue relates to the Podman, here is a new issue to their repository: https://github.com/containers/podman/issues/18916