Azure / doAzureParallel

A R package that allows users to submit parallel workloads in Azure
MIT License
107 stars 51 forks source link

Change MRAN Snapshot Date #51

Closed dustindall closed 6 years ago

dustindall commented 7 years ago

The MRAN snapshot date has been interfering in my ability to install the latest packages. I'd like to be able to set the snapshot date in the configuration file. I created a simple fix by changing the snapshot date before each package is installed (see below).

Rscript -e 'args <- commandArgs(TRUE)' -e 'options(repos = c(CRAN = args[1]))' https://mran.revolutionanalytics.com/snapshot/YYYY-MM-DD -e 'install.packages(args[1], dependencies=TRUE)' devtools

There are other ways to go about this, but if you like my solution let me know and I'll send a pull request.

Thanks, Dustin

paselem commented 7 years ago

@dustindall This looks pretty interesting. How do you propose you will expose this in the config?

dustindall commented 7 years ago

I would call it mranSnapshotDate and place it with rPackages before or after cran.

paselem commented 7 years ago

Yes, I think it makes sense to expose the snapshot date as it's own parameter. We are thinking about how to solve this generically going forward, as it seems we may want to expose repo-specific properties for each repo we support.

One proposed suggestion that seems to resonate well is:

{
  cran: {
    packages: []
  },
  mran:{
    packages:[],
    snapshotDate: 'latest | YYYY-MM-DD'
  },
  github: {
    packages: [],
    authenticationToken: 'empty | MY_GITHUB_AUTHENTICATION_TOKEN',
  }
}

We are thinking to expose this as a new way to structure the config in the next few releases - probably after milestone 0.5.0.

dustindall commented 7 years ago

Looks good. Thanks.

paselem commented 7 years ago

@dustindall - With the latest release of 0.4.0 we support custom scripts which allows you to run custom code at cluster setup time. You can use it to manually install the MRAN Snapshots you need.

dustindall commented 7 years ago

Awesome, I'll test it out and report back.

Thanks

paselem commented 6 years ago

I am closing this issue out since we now support Open Source R (CRAN-R) and give you the ability to use CRAN directly OR bring your own version of Microsoft R Open with a version to match the snapshots you want.