search

rocker-org / rocker-versioned

Run current & prior versions of R using docker
https://hub.docker.com/r/rocker/r-ver
GNU General Public License v2.0
297 stars 169 forks source link

using mran snapshots for packages #4

Closed kforner closed 7 years ago

kforner commented 10 years ago

The debian snapshot stuff is really great, and I guess we would need the same system for R packages. Is not that what MRAN snapshots are providing ? (only from the august 2014 currently)

cboettig commented 10 years ago

@kforner Interesting, hadn't heard of MRAN. I was able to find http://mran.revolutionanalytics.com/ but couldn't figure out how the snaphots work, but would be interested in a PR with an example.

Meanwhile, did you see #1 and this blog post? My understanding is that @gmbecker is currently working on a way to install old R packages given earlier versions of R. He knows this stuff way better than I so I'll let you two take it from here ;-)

eddelbuettel commented 10 years ago

As this appears to be a production-level issue for @kforner, I look forward to him taking the lead here on the actual implementation [/me flashes wide grin]

cmosetick commented 9 years ago

I'm one of the maintainers of MRAN. I'd like to provide assistance on this issue. The overview of how MRAN incorporates checkpoint-server snapshots is here: http://projects.revolutionanalytics.com/documents/rrt/rrtpkgs/

To get started actually using a point-in-time snapshot, the easiest thing to do is have a rocker container with checkpoint R package installed by default. Then instruct users to run:

library("checkpoint")`
## now load a relevant snapshot date
checkpoint("2014-12-01")

Otherwise the simplest way to incorporate snapshots is to set the container wide repository mirror to http://mran.revolutionanalytics.com/snapshot/<year-month-day> e.g. echo 'options(repos = list(CRAN = "http://mran.revolutionanalytics.com/snapshot/2014-12-17/"))' >> /etc/R/Rprofile.site

kforner commented 9 years ago

Thanks Chris. I currently use the option #2. Do you have any plans to add older MRAN snapshots ? I mean older than july 2014 or so.

My use case if that I'd like to provide a e.g. rocker R-3.0.2 with a corresponding snapshot. Using CRANDB it should be quite feasible from what I understand.

Best, Karl

On Wed, Dec 17, 2014 at 10:52 AM, Chris Mosetick notifications@github.com wrote:

I'm one of the maintainers of MRAN. I'd like to provide assistance on this issue. The overview of how MRAN incorporates checkpoint-server snapshots is here: http://projects.revolutionanalytics.com/documents/rrt/rrtpkgs/

To get started actually using a point-in-time snapshot, the easiest thing to do is have a rocker container with checkpoint http://mran.revolutionanalytics.com/packages/info/?checkpoint R package installed by default. Then instruct users to run:

library("checkpoint")`

now load a relevant snapshot date

checkpoint("2014-12-01")

Otherwise the simplest way to incorporate snapshots is to set the container wide repository mirror to http://mran.revolutionanalytics.com/snapshot/ e.g. echo 'options(repos = list(CRAN = " http://mran.revolutionanalytics.com/snapshot/2014-12-17/"))' >> /etc/R/Rprofile.site

— Reply to this email directly or view it on GitHub https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67299654 .

gmbecker commented 9 years ago

Karl,

See the code I added to the repository to use switchr ( http://github.com/gmbecker/switchr) to install packages from specific dates/R versions without the existence of a specific mran snapshot using crandb.

There will be a convenience function which wraps this functionality in (near) future versions of switchr.

I'm happy to answer any questions about what it is doing.

~G

On Wed, Dec 17, 2014 at 1:56 AM, Karl Forner notifications@github.com wrote:

Thanks Chris. I currently use the option #2. Do you have any plans to add older MRAN snapshots ? I mean older than july 2014 or so.

My use case if that I'd like to provide a e.g. rocker R-3.0.2 with a corresponding snapshot. Using CRANDB it should be quite feasible from what I understand.

Best, Karl

On Wed, Dec 17, 2014 at 10:52 AM, Chris Mosetick notifications@github.com

wrote:

I'm one of the maintainers of MRAN. I'd like to provide assistance on this issue. The overview of how MRAN incorporates checkpoint-server snapshots is here: http://projects.revolutionanalytics.com/documents/rrt/rrtpkgs/

To get started actually using a point-in-time snapshot, the easiest thing to do is have a rocker container with checkpoint http://mran.revolutionanalytics.com/packages/info/?checkpoint R package installed by default. Then instruct users to run:

library("checkpoint")`

now load a relevant snapshot date

checkpoint("2014-12-01")

Otherwise the simplest way to incorporate snapshots is to set the container wide repository mirror to http://mran.revolutionanalytics.com/snapshot/ e.g. echo 'options(repos = list(CRAN = " http://mran.revolutionanalytics.com/snapshot/2014-12-17/"))' >> /etc/R/Rprofile.site

— Reply to this email directly or view it on GitHub < https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67299654>

.

— Reply to this email directly or view it on GitHub https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67300088 .

Gabriel Becker Graduate Student Statistics Department University of California, Davis

kforner commented 9 years ago

Thanks Gabe, but I'm not sure this fulfills my use case. Suppose I provide a R-3.0.2 install. I could using switchr or another ways provide a set of packages with adequate versions. But suppose that at one point the user needs another package, and wants to install it. The odds are high that he will get: "package Foo requires R>3.1.1"

The advantage of a MRAN snapshot , with a date well fitted for the particular R version, is that it mostly solves this issue, in a very elegant way. And it is not at all incompatible with switchr.

On Wed, Dec 17, 2014 at 4:00 PM, Gabe Becker notifications@github.com wrote:

Karl,

See the code I added to the repository to use switchr ( http://github.com/gmbecker/switchr) to install packages from specific dates/R versions without the existence of a specific mran snapshot using crandb.

There will be a convenience function which wraps this functionality in (near) future versions of switchr.

I'm happy to answer any questions about what it is doing.

~G

On Wed, Dec 17, 2014 at 1:56 AM, Karl Forner notifications@github.com wrote:

Thanks Chris. I currently use the option #2. Do you have any plans to add older MRAN snapshots ? I mean older than july 2014 or so.

My use case if that I'd like to provide a e.g. rocker R-3.0.2 with a corresponding snapshot. Using CRANDB it should be quite feasible from what I understand.

Best, Karl

On Wed, Dec 17, 2014 at 10:52 AM, Chris Mosetick < notifications@github.com>

wrote:

I'm one of the maintainers of MRAN. I'd like to provide assistance on this issue. The overview of how MRAN incorporates checkpoint-server snapshots is here: http://projects.revolutionanalytics.com/documents/rrt/rrtpkgs/

To get started actually using a point-in-time snapshot, the easiest thing to do is have a rocker container with checkpoint http://mran.revolutionanalytics.com/packages/info/?checkpoint R package installed by default. Then instruct users to run:

library("checkpoint")`

now load a relevant snapshot date

checkpoint("2014-12-01")

Otherwise the simplest way to incorporate snapshots is to set the container wide repository mirror to http://mran.revolutionanalytics.com/snapshot/ e.g. echo 'options(repos = list(CRAN = " http://mran.revolutionanalytics.com/snapshot/2014-12-17/"))' >> /etc/R/Rprofile.site

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67299654>

.

— Reply to this email directly or view it on GitHub < https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67300088>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67333796 .

gmbecker commented 9 years ago

Karl,

I'm not sure I follow. switchr, via crandb, can install package versions from specific dates as well as from R versions. All switchr needs is a list of package versions and locations, and it can install any version of any (installable) package.

With some more work (which would have to be manual for now but could be a convenience function in either switchr or crandb later) it could take a package version, resolve that to a date, and install other packages (including dependencies for the selected package) from a "virtual snapshot" of that date provided by crandb to get a consistent cohort.

The only difference between switchr and mran for this use-case is where and when the packages are built, and whether a repository that 'physically' contains the files needs to exist before the process starts or not.

All switchr is doing is building mran snapshots (or, more precisely, subsets thereof) on the fly as necessary using info from crandb.

~G

On Wed, Dec 17, 2014 at 7:06 AM, Karl Forner notifications@github.com wrote:

Thanks Gabe, but I'm not sure this fulfills my use case. Suppose I provide a R-3.0.2 install. I could using switchr or another ways provide a set of packages with adequate versions. But suppose that at one point the user needs another package, and wants to install it. The odds are high that he will get: "package Foo requires R>3.1.1"

The advantage of a MRAN snapshot , with a date well fitted for the particular R version, is that it mostly solves this issue, in a very elegant way. And it is not at all incompatible with switchr.

On Wed, Dec 17, 2014 at 4:00 PM, Gabe Becker notifications@github.com wrote:

Karl,

See the code I added to the repository to use switchr ( http://github.com/gmbecker/switchr) to install packages from specific dates/R versions without the existence of a specific mran snapshot using crandb.

There will be a convenience function which wraps this functionality in (near) future versions of switchr.

I'm happy to answer any questions about what it is doing.

~G

On Wed, Dec 17, 2014 at 1:56 AM, Karl Forner notifications@github.com wrote:

Thanks Chris. I currently use the option #2. Do you have any plans to add older MRAN snapshots ? I mean older than july 2014 or so.

My use case if that I'd like to provide a e.g. rocker R-3.0.2 with a corresponding snapshot. Using CRANDB it should be quite feasible from what I understand.

Best, Karl

On Wed, Dec 17, 2014 at 10:52 AM, Chris Mosetick < notifications@github.com>

wrote:

I'm one of the maintainers of MRAN. I'd like to provide assistance on this issue. The overview of how MRAN incorporates checkpoint-server snapshots is here: http://projects.revolutionanalytics.com/documents/rrt/rrtpkgs/

To get started actually using a point-in-time snapshot, the easiest thing to do is have a rocker container with checkpoint http://mran.revolutionanalytics.com/packages/info/?checkpoint R package installed by default. Then instruct users to run:

library("checkpoint")`

now load a relevant snapshot date

checkpoint("2014-12-01")

Otherwise the simplest way to incorporate snapshots is to set the container wide repository mirror to http://mran.revolutionanalytics.com/snapshot/ e.g. echo 'options(repos = list(CRAN = " http://mran.revolutionanalytics.com/snapshot/2014-12-17/"))' >> /etc/R/Rprofile.site

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67299654>

.

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67300088>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub < https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67333796>

.

— Reply to this email directly or view it on GitHub https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67334706 .

Gabriel Becker Graduate Student Statistics Department University of California, Davis

kforner commented 9 years ago

I'm not sure I follow. switchr, via crandb, can install package versions from specific dates as well as from R versions. All switchr needs is a list of package versions and locations, and it can install any version of any (installable) package.

I'm not sure either to have understood what switchr can do. In my use case there is no a priori list. At one point the user would want to install a package not known. can you do something like: switchr_install_package(pkg = 'Foo', date = get_cran_snapshot_date_for("3,0,2")) ? Which would automatically fetch the proper version of package Foo (i.e. last published for instance)) that is compatible with R 3.0.2 ? Knowing that all the dependencies of package Foo must also be compatible..

With some more work (which would have to be manual for now but could be a convenience function in either switchr or crandb later) it could take a package version, resolve that to a date, and install other packages (including dependencies for the selected package) from a "virtual snapshot" of that date provided by crandb to get a consistent cohort.

As I said, in my use case I do not have a package version, just a R version.

The only difference between switchr and mran for this use-case is where and when the packages are built, and whether a repository that 'physically' contains the files needs to exist before the process starts or not.

All switchr is doing is building mran snapshots (or, more precisely, subsets thereof) on the fly as necessary using info from crandb.

Ok. If I understand well in that case switchr would be extremely useful then. I'll have a look ASAP

~G

On Wed, Dec 17, 2014 at 7:06 AM, Karl Forner notifications@github.com wrote:

Thanks Gabe, but I'm not sure this fulfills my use case. Suppose I provide a R-3.0.2 install. I could using switchr or another ways provide a set of packages with adequate versions. But suppose that at one point the user needs another package, and wants to install it. The odds are high that he will get: "package Foo requires R>3.1.1"

The advantage of a MRAN snapshot , with a date well fitted for the particular R version, is that it mostly solves this issue, in a very elegant way. And it is not at all incompatible with switchr.

On Wed, Dec 17, 2014 at 4:00 PM, Gabe Becker notifications@github.com wrote:

Karl,

See the code I added to the repository to use switchr ( http://github.com/gmbecker/switchr) to install packages from specific dates/R versions without the existence of a specific mran snapshot using crandb.

There will be a convenience function which wraps this functionality in (near) future versions of switchr.

I'm happy to answer any questions about what it is doing.

~G

On Wed, Dec 17, 2014 at 1:56 AM, Karl Forner notifications@github.com

wrote:

Thanks Chris. I currently use the option #2. Do you have any plans to add older MRAN snapshots ? I mean older than july 2014 or so.

My use case if that I'd like to provide a e.g. rocker R-3.0.2 with a corresponding snapshot. Using CRANDB it should be quite feasible from what I understand.

Best, Karl

On Wed, Dec 17, 2014 at 10:52 AM, Chris Mosetick < notifications@github.com>

wrote:

I'm one of the maintainers of MRAN. I'd like to provide assistance on this issue. The overview of how MRAN incorporates checkpoint-server snapshots is here: http://projects.revolutionanalytics.com/documents/rrt/rrtpkgs/

To get started actually using a point-in-time snapshot, the easiest thing to do is have a rocker container with checkpoint http://mran.revolutionanalytics.com/packages/info/?checkpoint R package installed by default. Then instruct users to run:

library("checkpoint")`

now load a relevant snapshot date

checkpoint("2014-12-01")

Otherwise the simplest way to incorporate snapshots is to set the container wide repository mirror to http://mran.revolutionanalytics.com/snapshot/ e.g. echo 'options(repos = list(CRAN = " http://mran.revolutionanalytics.com/snapshot/2014-12-17/"))' >> /etc/R/Rprofile.site

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67299654>

.

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67300088>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67333796>

.

— Reply to this email directly or view it on GitHub < https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67334706>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67337468 .

gmbecker commented 9 years ago

Karl,

On Wed, Dec 17, 2014 at 8:30 AM, Karl Forner notifications@github.com wrote:

I'm not sure I follow. switchr, via crandb, can install package versions from specific dates as well as from R versions. All switchr needs is a list of package versions and locations, and it can install any version of any (installable) package.

I'm not sure either to have understood what switchr can do. In my use case there is no a priori list. At one point the user would want to install a package not known. can you do something like: switchr_install_package(pkg = 'Foo', date = get_cran_snapshot_date_for("3,0,2")) ? Which would automatically fetch the proper version of package Foo (i.e. last published for instance)) that is compatible with R 3.0.2 ? Knowing that all the dependencies of package Foo must also be compatible..

It can. It turns out I have already added the convenience function for this:

library(RJSONIO) thing = switchr:::rVersionManifest("3.0.2") --- Please select a CRAN mirror for use in this session --- thing A package manifest (PkgManifest object)

Contains 4643 packages and 0 dependency repositories

Packages: name type 1 "A3" "tarball" 2 "abc" "tarball" 3 "abcdeFBA" "tarball" 4 "ABCExtremes" "tarball" ... "..." "..." 4639 "zoeppritz" "tarball" 4640 "zoo" "tarball" 4641 "zooimage" "tarball" 4642 "zoom" "tarball" 4643 "zyp" "tarball"

head(manifest_df(thing)) name 1 A3 2 abc 3 abcdeFBA 4 ABCExtremes 5 ABCp2 6 abd url type 1 http://cran.rstudio.com/src/contrib/A3_0.9.2.tar.gz tarball 2 http://cran.r-project.org/src/contrib/Archive/abc/abc_1.7.tar.gz tarball 3 http://cran.rstudio.com/src/contrib/abcdeFBA_0.4.tar.gz tarball 4 http://cran.rstudio.com/src/contrib/ABCExtremes_1.0.tar.gz tarball 5 http://cran.rstudio.com/src/contrib/ABCp2_1.1.tar.gz tarball 6 http://cran.r-project.org/src/contrib/Archive/abd/abd_0.2-5.tar.gz tarball branch subdir extra 1 NA . NA 2 NA . NA 3 NA . NA 4 NA . NA 5 NA . NA 6 NA . NA

It's not exported yet, but easily can be. Notice that the manifest that is generated points to tarballs with exact locations, some of which are in the CRAN archive, and others of which are in the current CRAN (meaning they haven't been updated since the date associated with that R version number).

You can then call install_packages(, repos = ) and it will do the right thing (ignore the warnings)

switchTo("testtmp") Switched to the 'testtmp' computing environment. 29 packages are currently available. Packages installed in your site library ARE suppressed. To switch back to your previous environment type switchBack() _installpackages("zoo", repos = thing) Warning: unable to access index for repository /src/contrib trying _URL 'http://cran.r-project.org/src/contrib/Archive/zoo/zoo_1.7-10.tar.gz http://cran.r-project.org/src/contrib/Archive/zoo/zoo_1.7-10.tar.gz'_ Content type 'application/x-gzip' length 813020 bytes (793 Kb)

opened URL

downloaded 793 Kb

trying _URL 'http://cran.r-project.org/src/contrib/Archive/lattice/lattice_0.20-23.tar.gz http://cran.r-project.org/src/contrib/Archive/lattice/lattice_0.20-23.tar.gz'_ Content type 'application/x-gzip' length 346198 bytes (338 Kb)

opened URL

downloaded 338 Kb

Warning: unable to access index for repository /src/contrib Installing package into ‘~/.switchr/testtmp’ (as ‘lib’ is unspecified)

The downloaded source packages are in ‘/tmp/RtmpEAvDZb/downloaded_packages’ Warning: unable to access index for repository /src/contrib zoo "zoo" Warning message: In dir.create(repdir, recursive = TRUE) : '/tmp/RtmpEAvDZb/repo/src/contrib' already exists

This required a couple of small bugfixes which I have just checked into master with a version bump to 0.4.1 (which also exports the aforementioned rVersionManifest function).

~G

~G

On Wed, Dec 17, 2014 at 7:06 AM, Karl Forner notifications@github.com wrote:

Thanks Gabe, but I'm not sure this fulfills my use case. Suppose I provide a R-3.0.2 install. I could using switchr or another ways provide a set of packages with adequate versions. But suppose that at one point the user needs another package, and wants to install it. The odds are high that he will get: "package Foo requires R>3.1.1"

The advantage of a MRAN snapshot , with a date well fitted for the particular R version, is that it mostly solves this issue, in a very elegant way. And it is not at all incompatible with switchr.

On Wed, Dec 17, 2014 at 4:00 PM, Gabe Becker notifications@github.com

wrote:

Karl,

See the code I added to the repository to use switchr ( http://github.com/gmbecker/switchr) to install packages from specific dates/R versions without the existence of a specific mran snapshot using crandb.

There will be a convenience function which wraps this functionality in (near) future versions of switchr.

I'm happy to answer any questions about what it is doing.

~G

On Wed, Dec 17, 2014 at 1:56 AM, Karl Forner < notifications@github.com>

wrote:

Thanks Chris. I currently use the option #2. Do you have any plans to add older MRAN snapshots ? I mean older than july 2014 or so.

My use case if that I'd like to provide a e.g. rocker R-3.0.2 with a corresponding snapshot. Using CRANDB it should be quite feasible from what I understand.

Best, Karl

On Wed, Dec 17, 2014 at 10:52 AM, Chris Mosetick < notifications@github.com>

wrote:

I'm one of the maintainers of MRAN. I'd like to provide assistance on this issue. The overview of how MRAN incorporates checkpoint-server snapshots is here: http://projects.revolutionanalytics.com/documents/rrt/rrtpkgs/

To get started actually using a point-in-time snapshot, the easiest thing to do is have a rocker container with checkpoint http://mran.revolutionanalytics.com/packages/info/?checkpoint R package installed by default. Then instruct users to run:

library("checkpoint")`

now load a relevant snapshot date

checkpoint("2014-12-01")

Otherwise the simplest way to incorporate snapshots is to set the container wide repository mirror to http://mran.revolutionanalytics.com/snapshot/ e.g. echo 'options(repos = list(CRAN = " http://mran.revolutionanalytics.com/snapshot/2014-12-17/"))' >> /etc/R/Rprofile.site

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67299654>

.

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67300088>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67333796>

.

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67334706>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub < https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67337468>

.

— Reply to this email directly or view it on GitHub https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67349237 .

Gabriel Becker Graduate Student Statistics Department University of California, Davis

gmbecker commented 9 years ago

Karl,

Make that switchr v 0.4.2, which checks if you have RJSONIO (or another package which provides a fromJSON function) automatically instead of mysteriously dying.

~G

On Wed, Dec 17, 2014 at 8:56 AM, Gabriel Becker gmbecker@ucdavis.edu wrote:

Karl,

On Wed, Dec 17, 2014 at 8:30 AM, Karl Forner notifications@github.com wrote:

I'm not sure I follow. switchr, via crandb, can install package versions from specific dates as well as from R versions. All switchr needs is a list of package versions and locations, and it can install any version of any (installable) package.

I'm not sure either to have understood what switchr can do. In my use case there is no a priori list. At one point the user would want to install a package not known. can you do something like: switchr_install_package(pkg = 'Foo', date = get_cran_snapshot_date_for("3,0,2")) ? Which would automatically fetch the proper version of package Foo (i.e. last published for instance)) that is compatible with R 3.0.2 ? Knowing that all the dependencies of package Foo must also be compatible..

It can. It turns out I have already added the convenience function for this:

library(RJSONIO) thing = switchr:::rVersionManifest("3.0.2") --- Please select a CRAN mirror for use in this session --- thing A package manifest (PkgManifest object)

Contains 4643 packages and 0 dependency repositories

Packages: name type 1 "A3" "tarball" 2 "abc" "tarball" 3 "abcdeFBA" "tarball" 4 "ABCExtremes" "tarball" ... "..." "..." 4639 "zoeppritz" "tarball" 4640 "zoo" "tarball" 4641 "zooimage" "tarball" 4642 "zoom" "tarball" 4643 "zyp" "tarball"

head(manifest_df(thing)) name 1 A3 2 abc 3 abcdeFBA 4 ABCExtremes 5 ABCp2 6 abd url type 1 http://cran.rstudio.com/src/contrib/A3_0.9.2.tar.gz tarball 2 http://cran.r-project.org/src/contrib/Archive/abc/abc_1.7.tar.gz tarball 3 http://cran.rstudio.com/src/contrib/abcdeFBA_0.4.tar.gz tarball 4 http://cran.rstudio.com/src/contrib/ABCExtremes_1.0.tar.gz tarball 5 http://cran.rstudio.com/src/contrib/ABCp2_1.1.tar.gz tarball 6 http://cran.r-project.org/src/contrib/Archive/abd/abd_0.2-5.tar.gz tarball branch subdir extra 1 NA . NA 2 NA . NA 3 NA . NA 4 NA . NA 5 NA . NA 6 NA . NA

It's not exported yet, but easily can be. Notice that the manifest that is generated points to tarballs with exact locations, some of which are in the CRAN archive, and others of which are in the current CRAN (meaning they haven't been updated since the date associated with that R version number).

You can then call install_packages(, repos = ) and it will do the right thing (ignore the warnings)

switchTo("testtmp") Switched to the 'testtmp' computing environment. 29 packages are currently available. Packages installed in your site library ARE suppressed. To switch back to your previous environment type switchBack() _installpackages("zoo", repos = thing) Warning: unable to access index for repository /src/contrib trying _URL 'http://cran.r-project.org/src/contrib/Archive/zoo/zoo_1.7-10.tar.gz http://cran.r-project.org/src/contrib/Archive/zoo/zoo_1.7-10.tar.gz'_ Content type 'application/x-gzip' length 813020 bytes (793 Kb)

opened URL

downloaded 793 Kb

trying _URL 'http://cran.r-project.org/src/contrib/Archive/lattice/lattice_0.20-23.tar.gz http://cran.r-project.org/src/contrib/Archive/lattice/lattice_0.20-23.tar.gz'_ Content type 'application/x-gzip' length 346198 bytes (338 Kb)

opened URL

downloaded 338 Kb

Warning: unable to access index for repository /src/contrib Installing package into ‘~/.switchr/testtmp’ (as ‘lib’ is unspecified)

  • installing source package ‘zoo’ ... * libs gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c coredata.c -o coredata.o gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c init.c -o init.o gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c lag.c -o lag.o gcc -std=gnu99 -shared -Wl,-Bsymbolic-functions -Wl,-z,relro -o zoo.so coredata.o init.o lag.o -L/usr/lib/R/lib -lR installing to /home/beckerg4/.switchr/testtmp/zoo/libs * R * demo * inst * preparing package for lazy loading * help ** installing help indices * building package indices * installing vignettes * testing if installed package can be loaded * DONE (zoo)

The downloaded source packages are in ‘/tmp/RtmpEAvDZb/downloaded_packages’ Warning: unable to access index for repository /src/contrib zoo "zoo" Warning message: In dir.create(repdir, recursive = TRUE) : '/tmp/RtmpEAvDZb/repo/src/contrib' already exists

This required a couple of small bugfixes which I have just checked into master with a version bump to 0.4.1 (which also exports the aforementioned rVersionManifest function).

~G

~G

On Wed, Dec 17, 2014 at 7:06 AM, Karl Forner notifications@github.com

wrote:

Thanks Gabe, but I'm not sure this fulfills my use case. Suppose I provide a R-3.0.2 install. I could using switchr or another ways provide a set of packages with adequate versions. But suppose that at one point the user needs another package, and wants to install it. The odds are high that he will get: "package Foo requires R>3.1.1"

The advantage of a MRAN snapshot , with a date well fitted for the particular R version, is that it mostly solves this issue, in a very elegant way. And it is not at all incompatible with switchr.

On Wed, Dec 17, 2014 at 4:00 PM, Gabe Becker < notifications@github.com> wrote:

Karl,

See the code I added to the repository to use switchr ( http://github.com/gmbecker/switchr) to install packages from specific dates/R versions without the existence of a specific mran snapshot using crandb.

There will be a convenience function which wraps this functionality in (near) future versions of switchr.

I'm happy to answer any questions about what it is doing.

~G

On Wed, Dec 17, 2014 at 1:56 AM, Karl Forner < notifications@github.com>

wrote:

Thanks Chris. I currently use the option #2. Do you have any plans to add older MRAN snapshots ? I mean older than july 2014 or so.

My use case if that I'd like to provide a e.g. rocker R-3.0.2 with a corresponding snapshot. Using CRANDB it should be quite feasible from what I understand.

Best, Karl

On Wed, Dec 17, 2014 at 10:52 AM, Chris Mosetick < notifications@github.com>

wrote:

I'm one of the maintainers of MRAN. I'd like to provide assistance on this issue. The overview of how MRAN incorporates checkpoint-server snapshots is here: http://projects.revolutionanalytics.com/documents/rrt/rrtpkgs/

To get started actually using a point-in-time snapshot, the easiest thing to do is have a rocker container with checkpoint http://mran.revolutionanalytics.com/packages/info/?checkpoint R package installed by default. Then instruct users to run:

library("checkpoint")`

now load a relevant snapshot date

checkpoint("2014-12-01")

Otherwise the simplest way to incorporate snapshots is to set the container wide repository mirror to http://mran.revolutionanalytics.com/snapshot/ e.g. echo 'options(repos = list(CRAN = " http://mran.revolutionanalytics.com/snapshot/2014-12-17/"))'

/etc/R/Rprofile.site

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67299654>

.

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67300088>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67333796>

.

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67334706>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub < https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67337468>

.

— Reply to this email directly or view it on GitHub https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67349237 .

Gabriel Becker Graduate Student Statistics Department University of California, Davis

Gabriel Becker Graduate Student Statistics Department University of California, Davis

gmbecker commented 9 years ago

Karl,

FYI, I spent some time updating the documentation of switchr over the winter break, so the helpfiles should be much more ... helpful in version 0.4.4 on Github. A comprehensive vignette should be forthcoming soon(ish) but if you have any questions in the meantime please don't hesitate to ask.

~G

On Wed, Dec 17, 2014 at 9:02 AM, Gabriel Becker gmbecker@ucdavis.edu wrote:

Karl,

Make that switchr v 0.4.2, which checks if you have RJSONIO (or another package which provides a fromJSON function) automatically instead of mysteriously dying.

~G

On Wed, Dec 17, 2014 at 8:56 AM, Gabriel Becker gmbecker@ucdavis.edu wrote:

Karl,

On Wed, Dec 17, 2014 at 8:30 AM, Karl Forner notifications@github.com wrote:

I'm not sure I follow. switchr, via crandb, can install package versions from specific dates as well as from R versions. All switchr needs is a list of package versions and locations, and it can install any version of any (installable) package.

I'm not sure either to have understood what switchr can do. In my use case there is no a priori list. At one point the user would want to install a package not known. can you do something like: switchr_install_package(pkg = 'Foo', date = get_cran_snapshot_date_for("3,0,2")) ? Which would automatically fetch the proper version of package Foo (i.e. last published for instance)) that is compatible with R 3.0.2 ? Knowing that all the dependencies of package Foo must also be compatible..

It can. It turns out I have already added the convenience function for this:

library(RJSONIO) thing = switchr:::rVersionManifest("3.0.2") --- Please select a CRAN mirror for use in this session --- thing A package manifest (PkgManifest object)

Contains 4643 packages and 0 dependency repositories

Packages: name type 1 "A3" "tarball" 2 "abc" "tarball" 3 "abcdeFBA" "tarball" 4 "ABCExtremes" "tarball" ... "..." "..." 4639 "zoeppritz" "tarball" 4640 "zoo" "tarball" 4641 "zooimage" "tarball" 4642 "zoom" "tarball" 4643 "zyp" "tarball"

head(manifest_df(thing)) name 1 A3 2 abc 3 abcdeFBA 4 ABCExtremes 5 ABCp2 6 abd url type 1 http://cran.rstudio.com/src/contrib/A3_0.9.2.tar.gz tarball 2 http://cran.r-project.org/src/contrib/Archive/abc/abc_1.7.tar.gz tarball 3 http://cran.rstudio.com/src/contrib/abcdeFBA_0.4.tar.gz tarball 4 http://cran.rstudio.com/src/contrib/ABCExtremes_1.0.tar.gz tarball 5 http://cran.rstudio.com/src/contrib/ABCp2_1.1.tar.gz tarball 6 http://cran.r-project.org/src/contrib/Archive/abd/abd_0.2-5.tar.gz tarball branch subdir extra 1 NA . NA 2 NA . NA 3 NA . NA 4 NA . NA 5 NA . NA 6 NA . NA

It's not exported yet, but easily can be. Notice that the manifest that is generated points to tarballs with exact locations, some of which are in the CRAN archive, and others of which are in the current CRAN (meaning they haven't been updated since the date associated with that R version number).

You can then call install_packages(, repos = ) and it will do the right thing (ignore the warnings)

switchTo("testtmp") Switched to the 'testtmp' computing environment. 29 packages are currently available. Packages installed in your site library ARE suppressed. To switch back to your previous environment type switchBack() _installpackages("zoo", repos = thing) Warning: unable to access index for repository /src/contrib trying _URL 'http://cran.r-project.org/src/contrib/Archive/zoo/zoo_1.7-10.tar.gz http://cran.r-project.org/src/contrib/Archive/zoo/zoo_1.7-10.tar.gz'_ Content type 'application/x-gzip' length 813020 bytes (793 Kb)

opened URL

downloaded 793 Kb

trying _URL 'http://cran.r-project.org/src/contrib/Archive/lattice/lattice_0.20-23.tar.gz http://cran.r-project.org/src/contrib/Archive/lattice/lattice_0.20-23.tar.gz'_ Content type 'application/x-gzip' length 346198 bytes (338 Kb)

opened URL

downloaded 338 Kb

Warning: unable to access index for repository /src/contrib Installing package into ‘~/.switchr/testtmp’ (as ‘lib’ is unspecified)

  • installing source package ‘zoo’ ... * libs gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c coredata.c -o coredata.o gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c init.c -o init.o gcc -std=gnu99 -I/usr/share/R/include -DNDEBUG -fpic -g -O2 -fstack-protector --param=ssp-buffer-size=4 -Wformat -Werror=format-security -D_FORTIFY_SOURCE=2 -g -c lag.c -o lag.o gcc -std=gnu99 -shared -Wl,-Bsymbolic-functions -Wl,-z,relro -o zoo.so coredata.o init.o lag.o -L/usr/lib/R/lib -lR installing to /home/beckerg4/.switchr/testtmp/zoo/libs * R * demo * inst * preparing package for lazy loading * help ** installing help indices * building package indices * installing vignettes * testing if installed package can be loaded * DONE (zoo)

The downloaded source packages are in ‘/tmp/RtmpEAvDZb/downloaded_packages’ Warning: unable to access index for repository /src/contrib zoo "zoo" Warning message: In dir.create(repdir, recursive = TRUE) : '/tmp/RtmpEAvDZb/repo/src/contrib' already exists

This required a couple of small bugfixes which I have just checked into master with a version bump to 0.4.1 (which also exports the aforementioned rVersionManifest function).

~G

~G

On Wed, Dec 17, 2014 at 7:06 AM, Karl Forner notifications@github.com

wrote:

Thanks Gabe, but I'm not sure this fulfills my use case. Suppose I provide a R-3.0.2 install. I could using switchr or another ways provide a set of packages with adequate versions. But suppose that at one point the user needs another package, and wants to install it. The odds are high that he will get: "package Foo requires R>3.1.1"

The advantage of a MRAN snapshot , with a date well fitted for the particular R version, is that it mostly solves this issue, in a very elegant way. And it is not at all incompatible with switchr.

On Wed, Dec 17, 2014 at 4:00 PM, Gabe Becker < notifications@github.com> wrote:

Karl,

See the code I added to the repository to use switchr ( http://github.com/gmbecker/switchr) to install packages from specific dates/R versions without the existence of a specific mran snapshot using crandb.

There will be a convenience function which wraps this functionality in (near) future versions of switchr.

I'm happy to answer any questions about what it is doing.

~G

On Wed, Dec 17, 2014 at 1:56 AM, Karl Forner < notifications@github.com>

wrote:

Thanks Chris. I currently use the option #2. Do you have any plans to add older MRAN snapshots ? I mean older than july 2014 or so.

My use case if that I'd like to provide a e.g. rocker R-3.0.2 with a corresponding snapshot. Using CRANDB it should be quite feasible from what I understand.

Best, Karl

On Wed, Dec 17, 2014 at 10:52 AM, Chris Mosetick < notifications@github.com>

wrote:

I'm one of the maintainers of MRAN. I'd like to provide assistance on this issue. The overview of how MRAN incorporates checkpoint-server snapshots is here: http://projects.revolutionanalytics.com/documents/rrt/rrtpkgs/

To get started actually using a point-in-time snapshot, the easiest thing to do is have a rocker container with checkpoint http://mran.revolutionanalytics.com/packages/info/?checkpoint R package installed by default. Then instruct users to run:

library("checkpoint")`

now load a relevant snapshot date

checkpoint("2014-12-01")

Otherwise the simplest way to incorporate snapshots is to set the container wide repository mirror to http://mran.revolutionanalytics.com/snapshot/ e.g. echo 'options(repos = list(CRAN = " http://mran.revolutionanalytics.com/snapshot/2014-12-17/"))'

/etc/R/Rprofile.site

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67299654>

.

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67300088>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67333796>

.

— Reply to this email directly or view it on GitHub <

https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67334706>

.

Gabriel Becker Graduate Student Statistics Department University of California, Davis

— Reply to this email directly or view it on GitHub < https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67337468>

.

— Reply to this email directly or view it on GitHub https://github.com/rocker-org/rocker-versioned/issues/4#issuecomment-67349237 .

Gabriel Becker Graduate Student Statistics Department University of California, Davis

Gabriel Becker Graduate Student Statistics Department University of California, Davis

Gabriel Becker, PhD Alumnus Statistics Department University of California, Davis

cboettig commented 7 years ago

We now use MRAN snapshots. These are fixed by image in /etc/environment as the env var MRAN. For old R images the MRAN snapshot is set as the default CRAN mirror, while the current image uses the rstudio CRAN mirror.