search

insightsengineering / nestdevs-tasks

0 stars 0 forks source link

[Question]: Anyway we can setup r-universe so it keeps archives of packages? #55

Open m7pr opened 4 months ago

m7pr commented 4 months ago

I am working on setting up a pak .lockfile that would allow to reproduce the session in which teal shiny app is running. https://github.com/insightsengineering/coredev-tasks/issues/479 and I figured out we may want to setup an archive for r-universe.dev R archive network.

Imagine someone installed teal.reporter from https://pharmaverse.r-universe.dev/teal.reporter on a specific day and got version 0.3.1.9006. Then created a teal app and exposed it with a possibility to see session info and pack .lockfile.

Now imagine we merge features and fixes to main branch on https://github.com/insightsengineering/teal.reporter/ repository. This means that teal.reporter gets a vbump and a new development version is pushed to https://pharmaverse.r-universe.dev/teal.reporter and the old one 0.3.1.9006 is removed (from https://pharmaverse.r-universe.dev/teal.reporter), because we do not keep track of previous packages (we do not keep the archive of packages).

It is then impossible to restore the session from the mentioned teal app that is based on teal.reporter 0.3.1.9006 because r-universe no longer have this version. pak has no way to figure out that you can also install teal.reporter from Github because the session info and the .lockfile present the source of teal.reporter installation which is https://pharmaverse.r-universe.dev/teal.reporter

m7pr commented 4 months ago

CC @pawelru @gogonzo @donyunardi

m7pr commented 4 months ago

Btw a hilarious Details information from pak::lockfile_create

image
pawelru commented 4 months ago

To the best of my knowledge, this is not possible by design and there is nothing we can do about that. That's a known limitation of creating snapshot consisting of packages installed from r-universe. The only solution I'm seeing is installing from long-persisting sources such as GH or CRAN / CRAN-like repos.

m7pr commented 4 months ago

What if we support drat-like archive, and keep pushing new releases? From the user perspective it's an install.packages call but with a different repos argument which is copied anyway. Òn our end we would need to setup:

In the current setup r-universe does not bring much value besides testing on multiple platforms. It does not support reproducibility.

pawelru commented 4 months ago

We used to have this some time ago. It's there: https://github.com/insightsengineering/depository

This was abandoned for the reasons I cannot recall I am afraid. @cicdguy do you happen to remember what was bad about this?

In the current setup r-universe does not bring much value besides testing on multiple platforms. It does not support reproducibility.

There are number of features that are quite cool actually - binaries for wasm or binaries for devel state of package to faster install deps in CI. For reproducibility you are right. That's why it needs to be used only when repro is not required. We knew this from the very beginning.

m7pr commented 4 months ago

yeah, I don't disagree or anything. I think it's just a time to think about a repository/archive that also supports past versions of packages

vedhav commented 4 months ago

I think this will never be possible using r-universe. Also, it's going to be an overkill. The only feasible way to achieve dev package install is by building them from source from the GitHub repos.

vedhav commented 4 months ago

Back in the day, we were hitting API rate limits because all the repos were being downloaded from GiHub. But, now with the CRAN release this should not be the case if only few packages are being installed from GitHub API. I think this was also the problem with depository.

m7pr commented 4 months ago

Anyway we can setup r-universe so it keeps archives of packages?

This is not the real question honestly. This is just a nudge to move towards a solution that supports having past versions of packages. drat helps creating such networks/archives

vedhav commented 4 months ago

This is not the real question honestly.

haha of course! With some money we can do anything we want 🤑. The closest thing I can think of is to get a PPM and release on main merge like we do in our internal package manager.