search

rocker-org / rocker-versioned2

Run current & prior versions of R using docker. rocker/r-ver, rocker/rstudio, rocker/shiny, rocker/tidyverse, and so on.
https://rocker-project.org
GNU General Public License v2.0
419 stars 180 forks source link

Upgrade path for ml-verse (moving to R v 4.4)? #806

Closed mskyttner closed 6 months ago

mskyttner commented 6 months ago

Container image name

rocker/ml-verse:4.3.2

Container image digest

No response

What operating system related to this question?

Linux

System information

Question

I noticed an R vulnerability which seems to be mitigated if using R v 4.4.

In conjunction with making some updates of an image which is based on and extends ml-verse:4.3.2, I was therefore planning initially just to switch it to start off with rocker/ml-verse:4.4 but noticed that the ml-verse images are no longer updated...

I wonder if you have any advice or suggestions on the upgrade path I should take for moving to "rocker/ml-verse:4.4"?

I also wanted to ask about the images provided from ghcr.io (rather than the docker hub) - figuring using those from GitHub runners might provide some slight latency advantage when spun up in GHAs. Are the most recent variants of the rocker images served from docker hub or are the ones in the GitHub Container Registry equally "up-to-date"?

eitsupi commented 6 months ago

Sorry for bothering you. I recently did a major rewrite of the repository and have not yet succeeded in building rocker/ml-verse. Hopefully we will have a successful build by the end of the day.

I noticed an R vulnerability which seems to be mitigated if using R v 4.4.

I suggest you rethink whether that is really a reason to start using R4.4.0. https://github.com/hrbrmstr/rdaradar

figuring using those from GitHub runners might provide some slight latency advantage when spun up in GHAs. Are the most recent variants of the rocker images served from docker hub or are the ones in the GitHub Container Registry equally "up-to-date"?

Yes.

eitsupi commented 6 months ago

I fixed the CI and triggered a build, but unfortunately it seems to be unable to build due to a bad connection to CTAN. See the log https://github.com/rocker-org/rocker-versioned2/actions/runs/8985639266/job/24681391070

@cboettig I have seen too many build failures caused by latex. Do you have any suggestions for a solution? I think it would be better to copy the contents of rocker/verse etc. in a multi-stage build to reduce the number of times such a high probability of failure step is executed.

mskyttner commented 6 months ago

@eitsupi thanks for the update and the advice! Thanks for de-hyping that vuln, it doesn't appear to be too scary. There are other better reasons I guess for using R v 4.4, perhaps including the initially fuzzy feeling of being up-to-date and being able to support use of the fancy new %||% operator etc :). I see now ghcr.io/rocker-org/ml-verse:4.4.0, thanks so much!

benz0li commented 6 months ago

I have seen too many build failures caused by latex.

@eitsupi Is it a bad connection to a CTAN mirror? My builds sometimes fail because of

tlmgr: Remote database (revision 71082 of the texlive-scripts package) seems to be older than the local installation (rev 71089 of texlive-scripts); please use a different mirror and/or wait a day or two.

Do you have any suggestions for a solution?

I simply set retry: 2 in my .gitlab-ci.yml

eitsupi commented 6 months ago

I see now ghcr.io/rocker-org/ml-verse:4.4.0, thanks so much!

Sorry, That tag is wrong. See #810. (And thanks for making me aware that it had been pushed.)

eitsupi commented 6 months ago

I simply set retry: 2 in my .gitlab-ci.yml

Thanks, but I failed twice today, so retries don't seem to make sense to me.

eitsupi commented 6 months ago

This is the third time today that I have failed. Will not go any further, there seems to be a problem with the CTAN mirror.

https://github.com/rocker-org/rocker-versioned2/actions/runs/8987434399/job/24685907698#step:7:3077

#55 68.77 --2024-05-07 14:47:17--  https://mirror.ctan.org/systems/texlive/tlnet/install-tl-unx.tar.gz
#55 68.78 Resolving mirror.ctan.org (mirror.ctan.org)... 89.58.7.101
#55 83.79 Connecting to mirror.ctan.org (mirror.ctan.org)|89.58.7.101|:443... connected.
#55 84.01 HTTP request sent, awaiting response... 307 Temporary Redirect
#55 84.23 Location: https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz [following]
#55 84.23 --2024-05-07 14:47:33--  https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
#55 84.23 Resolving ctan.math.washington.edu (ctan.math.washington.edu)... 128.95.224.254
#55 84.44 Connecting to ctan.math.washington.edu (ctan.math.washington.edu)|128.95.224.254|:443... connected.
#55 84.55 ERROR: cannot verify ctan.math.washington.edu's certificate, issued by ‘CN=InCommon RSA Server CA 2,O=Internet2,C=US’:
#55 84.55   Unable to locally verify the issuer's authority.
#55 84.55 To connect to ctan.math.washington.edu insecurely, use `--no-check-certificate'.
#55 ERROR: process "/bin/sh -c /rocker_scripts/install_verse.sh" did not complete successfully: exit code: 5
benz0li commented 6 months ago

This is the third time today that I have failed. Will not go any further, there seems to be a problem with the CTAN mirror.

No.

Open https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz in the browser.

eitsupi commented 6 months ago

This problem seems to reproduce on Ubuntu.

$ wget https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
--2024-05-07 15:10:10--  https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
Resolving ctan.math.washington.edu (ctan.math.washington.edu)... 128.95.224.254, 128.95.224.254
Connecting to ctan.math.washington.edu (ctan.math.washington.edu)|128.95.224.254|:443... connected.
ERROR: cannot verify ctan.math.washington.edu's certificate, issued by ‘CN=InCommon RSA Server CA 2,O=Internet2,C=US’:
  Unable to locally verify the issuer's authority.
To connect to ctan.math.washington.edu insecurely, use `--no-check-certificate'.
eddelbuettel commented 6 months ago

Maybe try wget --no-check-certificate ... ? Or install the ca-certificates package? (Both just guesses from here...)

benz0li commented 6 months ago

This problem seems to reproduce on Ubuntu.

Debian 12 (bookworm) with ca-certificates installed:

$ wget https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
--2024-05-07 17:13:40--  https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz
Resolving ctan.math.washington.edu (ctan.math.washington.edu)... 128.95.224.254
Connecting to ctan.math.washington.edu (ctan.math.washington.edu)|128.95.224.254|:443... connected.
ERROR: The certificate of ‘ctan.math.washington.edu’ is not trusted.
ERROR: The certificate of ‘ctan.math.washington.edu’ doesn't have a known issuer.

@eitsupi Report to webmaster@ctan.org, then.

(Luckily, my server is far away from Washington 😉)

Note: Please take care not to send any HTML mails to these addresses, because HTML mails are held in CTAN's SPAM filter, and it may take some time until a postmaster comes along to set them free.

– https://ctan.org/contact

cboettig commented 6 months ago

Yes the tex errors are because of CTAN being so unreliable. I really think we should scrap the entire manual tlmgr route and stick with installing texlive from the ubuntu repos instead.

Yeah it is a bit large but so much more reliable. (Maybe we can also re-evaluate which images need tex?)

(Apologies for sending reply from email while on the move)

Carl Boettiger http://carlboettiger.info/

On Tue, May 7, 2024 at 8:20 AM Olivier Benz @.***> wrote:

This problem seems to reproduce on Ubuntu.

Debian 12 (bookworm) with ca-certificates installed:

$ wget https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz --2024-05-07 https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz--2024-05-07 17:13:40-- https://ctan.math.washington.edu/tex-archive/systems/texlive/tlnet/install-tl-unx.tar.gz Resolving ctan.math.washington.edu (ctan.math.washington.edu)... 128.95.224.254 Connecting to ctan.math.washington.edu (ctan.math.washington.edu)|128.95.224.254|:443... connected. ERROR: The certificate of ‘ctan.math.washington.edu’ is not trusted. ERROR: The certificate of ‘ctan.math.washington.edu’ doesn't have a known issuer.

@eitsupi https://github.com/eitsupi Report to @.***, then.

(Luckily, my server is far away from Washington 😉)

— Reply to this email directly, view it on GitHub https://github.com/rocker-org/rocker-versioned2/issues/806#issuecomment-2098692957, or unsubscribe https://github.com/notifications/unsubscribe-auth/AABWK6TQ5LAH4A6PP3L3EP3ZBDWM7AVCNFSM6AAAAABHKQB5O2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDAOJYGY4TEOJVG4 . You are receiving this because you were mentioned.Message ID: @.***>

eitsupi commented 6 months ago

I sent an email to CTAN and the problem seems to be resolved. I triggered the build again.

eitsupi commented 6 months ago

A new build has been pushed, thanks all. https://github.com/rocker-org/rocker-versioned2/wiki/ml-verse_acca11003d86