Binary package installation only works after manually updating apt indices

[grantmcdermott]

The great selling point of r2u is that it installs R package binaries and resolves system dependencies. Unfortunately, this doesn't appear to be working quite as expected here.

The very first time I try to install an R package (e.g., install.packages("data.table")) after instantiating a codespace, it doesn't fetch the binaries and instead installs/builds from source. Example output:

r$> install.packages("data.table")
Install system packages as root...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Reading package lists... Done    
Building dependency tree... Done 
Reading state information... Done
Fetched 0 B in 0s (0 B/s)                                                                                                                                                             
Installing package into ‘/usr/local/lib/R/site-library’
(as ‘lib’ is unspecified)
trying URL 'https://cloud.r-project.org/src/contrib/data.table_1.14.8.tar.gz'
Content type 'application/x-gzip' length 5338582 bytes (5.1 MB)
==================================================
downloaded 5.1 MB

* installing *source* package ‘data.table’ ...
<etc>

However.... What's even stranger is that once I q() this first R session and open a new one (e.g., by manually opening up an R terminal) then everything works as expected. Running install.packages("PKGNAME") installs the binary from the r2u mirror and resolves any system deps.

r$> install.packages("data.table")
Install system packages as root...
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Reading package lists... Done    
Building dependency tree... Done 
Reading state information... Done
Get:1 https://r2u.stat.illinois.edu/ubuntu jammy/main amd64 r-cran-data.table amd64 1.14.8-1.ca2204.1 [1833 kB]                                                                       
Fetched 1833 kB in 0s (0 B/s)                                                                                                                                                         
Selecting previously unselected package r-cran-data.table.
(Reading database ... 48038 files and directories currently installed.)
Preparing to unpack .../r-cran-data.table_1.14.8-1.ca2204.1_amd64.deb ...
Unpacking r-cran-data.table (1.14.8-1.ca2204.1) ...
Setting up r-cran-data.table (1.14.8-1.ca2204.1) ...

[grantmcdermott]

UPDATE: The below error message was fixed by https://github.com/grantmcdermott/codespaces-r2u/commit/ac8d3809e22ee95dbbc97fa1eca208b9dbcd9693 but the source installation problem remains....

---

(Original comment)

Is this the problem?

** testing if installed package can be loaded from final location
Warning message:
D-Bus service not found!

- If you are in a container environment, please consider adding the
  following to your configuration to silence this warning:

  options(bspm.sudo = TRUE)

- If you are in a desktop/server environment, please remove any 'bspm'
  installation from the user library and force a new system
  installation as follows:

  $ sudo Rscript --vanilla -e 'install.packages("bspm", repos="https://cran.r-project.org/")'

Note: I have set the options(bspm.sudo = TRUE) option here, so not sure why it still complains about that.

[grantmcdermott]

@eitsupi sorry to tag you out of the blue, but do you have any idea for the odd behaviour described above?

P.S. I should add that I'm experiencing the same problem when I clone the repo and open as a local DevContainer in VS Code.

[eddelbuettel]

We should probably also tag / invite @enchufa2 as this is a bspm misfeature. Which may or may not be bumbled by codespaces but @enchufa2 may know. Can you invite him? [Edit: I had missed that you made the repo public. ]

[eitsupi]

@eitsupi sorry to tag you out of the blue, but do you have any idea for the odd behaviour described above?

P.S. I should add that I'm experiencing the same problem when I clone the repo and open as a local DevContainer in VS Code.

I do not know. Perhaps there is a problem with loading Rprofile?

How did you open the R terminal? I know there are multiple ways to do this, but do you get the same problem with any of them?

[eddelbuettel]

Good @eitsupi and also what I tried when bumbling around in the container:

• same when starting R directly in terminal
• same when copying /etc/R/Rprofile.site to . and starting

It seems baked in at a higher level. It could also be that initialization is altered in its sequence but I do not see how that could happen in the fairly robust and tested model R has, It is a puzzle!

[grantmcdermott]

I do not know. Perhaps there is a problem with loading Rprofile?

How did you open the R terminal? I know there are multiple ways to do this, but do you get the same problem with any of them?

Just to add on top of of @eddelbuettel's comment: 1) I don't think that it's a problem with R profile loading, since it is recognizing at least some (all?) options that we specify there. 2) I have tested with the native (i.e., non-radian) R terminal and switching off all of the other optional features in devcontainer.json and still have the same problem. A puzzle, indeed!

[eddelbuettel]

(And it is easy, and informative, to add a quick cat("End of Rprofile.site\n") to it to confirm.)

[grantmcdermott]

Okay, have made some progress.

Rather than restarting your R session, running sudo apt update -qq in the terminal manually fixes things. This can be run at any time and it will immediately fix the problem, including with an existing R session open. Moreover, it appears to be a one-time fix since the correct behaviour persists thereafter, even when stopping and restarting a codespace/devcontainer instance.

I don't understand why updating the indices manually works here, since (a) we run this several times in Dockerfile and (b) I thought that bspm/r2u was meant to do this automatically. But it feels like we're closer to an out-of-the-box solution...

P.S. I tried appending a final RUN sudo apt update -qq at the end of the Dockerfile, but that doesn't work for me. I had to trigger it manually in the terminal after the initial build was completed. P.P.S. I thought it might be a user issue and reverted to the default "vscode" user for this image rather than creating a new "docker" user (https://github.com/grantmcdermott/codespaces-r2u/commit/141d95f5aef83662658a01b5c0ac1be4bd14c980). But again, this didn't seem to make a difference.

[eddelbuettel]

Yes I just sent you (via slack) progress report when using the same container locally. Works as is. So something is made ever so slightly different by devcontainers.

[eddelbuettel]

Confirming. The basic r2u magic works iff we run sudo apt update -qq first. Yet we also have the dual list files. Two of these are from your Dockerfile, two of them are not so we may need to have a look at the other scriptlets that run:

r$> system("ls -ltr /etc/apt/sources.list.d/")
total 16
-rw-r--r-- 1 root root 129 Aug  4 23:23 cran.list
-rw-r--r-- 1 root root 127 Aug  4 23:23 r2u.list
-rw-r--r-- 1 root root  76 Aug  4 23:24 cran-ubuntu.list
-rw-r--r-- 1 root root  65 Aug  4 23:24 cranapt.list

r$>

[grantmcdermott]

Thanks @eddelbuettel. Let's open up a separate issue for the dual list files.

[eddelbuettel]

It's merely a symptom of other configuration potentially getting in the way. So a "tell", not an issue per se.

[grantmcdermott]

Fixed with https://github.com/grantmcdermott/codespaces-r2u/commit/7bbe63ac777ca8ad84f186f5329211187b8659d4 (all hail @eitsupi!)

[Enchufa2]

Thanks for putting this together, looks great! Just catching up here. I understand from the conversation that the issue had to do with the apt lists somehow, and that there is no issue to track upstream in bspm, right?

[eddelbuettel]

I think the mimimal summary is Grant goofed my misordering bspm options and I goofed by not noticing, so we chased our tails. Setting options(bspm.FOO=...) after bspm::enable() is a non-op and we should have known.

All good, and exciting use cases of bspm + r2y inside devcontainers and codespaces mostly thanks to the unstoppable @eitsupi.