search

CredibilityLab / groundhog

Reproducible R Scripts Via Date Controlled Installing & Loading of CRAN & Git Packages
https://groundhogr.com/
GNU General Public License v3.0
78 stars 4 forks source link

MRAN shutting down #83

Closed aghaynes closed 1 year ago

aghaynes commented 1 year ago

As I understand it, groundhog currently looks for package binaries on MRAN if they're not on CRAN, before going back to CRAN for source... I guess you might be aware already, but MRAN is shutting down at the end of June... https://techcommunity.microsoft.com/t5/azure-sql-blog/microsoft-r-application-network-retirement/ba-p/3707161.

urisohn commented 1 year ago

Thanks. I just learned about this earlier today. Indeed, they are ending their snapshots (daily backups) end of the month Considering some promising alternatives at the moment. Thanks for the heads up.

Most likely will switch to R Studio's Posit, which started in 2017.

urisohn commented 1 year ago

I have a bit more of a plan now. 1) MRAN will stop doing snapshots on Jan 31st, 2023. Ideally by then a new version of groundohg (v2.2.0) will rely on R Studio's posit archive for binaries published on CRAN after that date. 2) MRAN will no longer make older snapshots available in July 2023. I will make a copy of all Windows and Mac binaries available on CRAN the 1st of each month, between 2015 and 2019 (when R Studio's posit started having more regular backups). These backups will be stored in an S3 bucket, most likely by researchbox.org, on a wasaby (S3 provider) server.
3) Around May/June of 2023 groundhog v2.3.0 will switch from MRAN to the researchbox repository for installing binaries from before 2019.

Plan subject to change of course.

aghaynes commented 1 year ago

Sounds like a good plan to me! 😄

urisohn commented 1 year ago

Update.

Grondhog v.2.2.0 was released as planned, and does not rely on MRAN past Jan 31st 2023. It still relies on MRAN for binaries prior to 2023. And for non-current packages released after January 2023 it relies on source installation, but this is a stop-gap situation The next release of groundhog will as planned rely on binaries past 2023 as well.

I did decide not to rely on Posit's backups, for they do not include Mac binaries and are not sufficiently transparent in terms of exactly what is being backed up and we would again be relying on a for-profit institution for the archiving of binaries (MRAN is by Microsoft, Posit is a for-profit), which makes me a bit nervous. They could discontinue at any point if no longer profitable to keep, or could restrict access, or whatever they'd like really.

Instead, i will rely on GRAN, a custom created archive of binaries originally available on CRAN (GRAN: groundhog R archive neighbor; picture asking a neighbor for sugar, but in this case you are asking for an old binary).

For 2014-2022 binaries in GRAN are based on binaries as available on MRAN. Starting with 2023, binaries for new packages released to CRAN are added to GRAN. GRAN is already available, and updates daily (S3 bucket) available at http://gran.grondhogr.com

Binaries are saved in a less redundat way than MRAN or Posit (where all binaries are saved again each day). Details will be provided later on a blog post and on grondhogr.com

Groundhog 2.3, or possibly 3.0, will be released in April/May 2023 and will download binaries for older packages from GRAN, for both windows and Mac. This should make groundog much faster that it is now, as MRAN is a bit slow. In adddition the new version of groundhog will move to parallel installation from source, with the goal of becoming the fastest option for installing packages, whether binary or source, new or archived. It will be faster than install.packages() and faster than relyingon Posit package manager. At least that's the goal. We'll see with the new release though.

urisohn commented 1 year ago

Groundhog 3.0.0 was released and is MRAN free.