pbiecek / archivist

A set of tools for datasets and plots archiving
http://pbiecek.github.io/archivist/
74 stars 9 forks source link

ainstall #251

Closed MarcinKosinski closed 8 years ago

MarcinKosinski commented 8 years ago

for a specific md5hash, asession gives packages and versions under which it was created, maybe one should create ainstall function that install specific versions of R packages with the usage of this package https://cran.r-project.org/web/packages/versions/index.html

pbiecek commented 8 years ago

Cool idea, but I am not sure if it's possible for github packages or own packages installed from source (tag.gz files).

2016-02-17 16:06 GMT+01:00 Marcin Kosiński notifications@github.com:

for a specific md5hash, asession gives packages and versions under which it was created, maybe one should create ainstall function that install specific versions of R packages with the usage of this package https://cran.r-project.org/web/packages/versions/index.html

— Reply to this email directly or view it on GitHub https://github.com/pbiecek/archivist/issues/251.

pozdrawiam serdecznie, Przemysław Biecek

MarcinKosinski commented 8 years ago

Or we can add a note to asession documentation about this feature.

MarcinKosinski commented 8 years ago

'versions' fits in the narrow gap between the 'devtools' install_version() function and the 'checkpoint' package. devtools::install_version() installs a stated package version from source files stored on the CRAN archives. However CRAN does not store binary versions of packages so Windows users need to have RTools installed and Windows and OSX users get longer installation times. 'checkpoint' uses the Revolution Analytics MRAN server to install packages (from source or binary) as they were available on a given date.

MarcinKosinski commented 8 years ago

255

MarcinKosinski commented 8 years ago

I went with devtools::install_version (but it does not support lib parameter passed to install.packages, so I also neededtouse .libPaths() :

check this out

# Recreate packages library for artifact
Marcin Kosiński  
27 lutego 2016  

# Retrieve object created with archive-version of ggplot2

```r
library(devtools)
install.packages('ggplot2', repos = 'https://cran.rstudio.com/')
library(ggplot2)
devtools::session_info(pkgs = 'ggplot2')
adir <- 'adir'
dir.create(adir)
.libPaths(c(adir, .libPaths())) # lib argument in devtools does not work
install.packages('devtools', repos = 'https://cran.rstudio.com/')
devtools::install_version('ggplot2', version = '1.0.1', lib = adir, repos = 'https://cran.rstudio.com/')
library(ggplot2, lib = adir)
devtools::session_info('ggplot2')

arecreate()

library(archivist)
system.file('graphGallery', package = 'archivist') -> archivist.dir
setLocalRepo(archivist.dir)
arecreate.dir <- 'arecreate.dir' 
dir.create('arecreate.dir')

arecreate <- function(md5hash = '600bda83cb840947976bd1ce3a11879d', lib, repos = 'https://cran.rstudio.com/' ){

  old_lib <- .libPaths()
  new_lib <- .libPaths(lib)

  # probably local installations or GitHub packages
  NOT_CRAN <- which(asession(md5hash)$packages[, '*'] == '*')

  CRAN_PKGS <- grep('CRAN',
                    x = asession(md5hash)$packages[, 'source'])
  GITHUB_PKGS <- grep('Github',
                    x = asession(md5hash)$packages[, 'source'])
  # reinstall CRAN packages

  sapply(setdiff(CRAN_PKGS, NOT_CRAN), function(package){
    devtools::install_version(asession(md5hash)$packages[package, 'package'],
                                 version = asession(md5hash)$packages[package, 'version'], 
                                 lib = lib, repos = repos )
  })
  # extract commits and names from parenthesis
  gsub("[\\(\\)]",
       "", 
       regmatches(asession(md5hash)$packages[GITHUB_PKGS, 'source'],
                  gregexpr("\\(.*?\\)",
                           asession(md5hash)$packages[GITHUB_PKGS, 'source'])
                  )[[1]]
       ) -> GITHUB_NAMES
    # reinstall GitHub packages
  devtools::install_github(GITHUB_NAMES, lib = lib)
  .libPaths(old_lib)
}
arecreate('600bda83cb840947976bd1ce3a11879d', lib = arecreate.dir)
knitr::kable(installed.packages(arecreate.dir))
pbiecek commented 8 years ago

There is a lot of calls to asession(md5hash) You can call this function just one time, and then use the results.

But larger problem is that it is not working on OSX ;-)

tar: Unrecognized archive format
tar: Error exit delayed from previous errors.
Error: file ‘/var/folders/_l/jllqh32s3llbxmtrh2431vpr0000gn/T//RtmpUP1PxN/downloaded_packages/assertthat_0.1.tgz’ is not an OS X binary package
In addition: Warning message:
'tar' returned non-zero exit code 1 

You need to add , type="source" to the install_version() function.

I will do more tests in a few days

pbiecek commented 8 years ago

I was thinking about other possible names fir this function. Maybe something that is relates with revert or restore` (since it reverts/restores previous versions of packages)?

MarcinKosinski commented 8 years ago

libraryRestore? arestore? So if it does not work for MAC OSX, so we'd better move to versions package when this https://github.com/goldingn/versions/issues/7 will be fixed

2016-02-28 23:47 GMT+01:00 Przemysław Biecek notifications@github.com:

I was thinking about other possible names fir this function. Maybe something that is relates with revert or restore` (since it reverts/restores previous versions of packages)?

— Reply to this email directly or view it on GitHub https://github.com/pbiecek/archivist/issues/251#issuecomment-189962290.

pbiecek commented 8 years ago

It will work, you just need to add type="source" to the install_version()

I like libraryRestore more, since the name is more self explanatory, or maybe restoreLibrary / revertLibrary, since in other functions the verb is first.

pbiecek commented 8 years ago

I've added restoreLibs() function. There are some dirty solutions, so I will think more about alternatives. But at least it is working for ggplot2 objects

MarcinKosinski commented 8 years ago

I've added devtools check - if it is installed - since we do not have imports from devtools https://github.com/pbiecek/archivist/commit/400216ced8f8f7df52cca966e0c0c0022cf1a43d

MarcinKosinski commented 8 years ago

By now restoreLibs downgrades artifacts in the first library of .libPaths() . Wouldn't it be a good idea to install old packages in a new directory, that would not change the state of packages used by user on a daily routine? I think additional parameter lib.loc could solve such demand or we could add additional information to manual page, that a user could manually change .libPaths() on his own.

If that's ok for you additional example to restoreLibs could look something like this


  old_lib <- .libPaths()
  new_lib <- .libPaths(new_directory)
  restoreLibs()
 .libPaths(old_lib)
pbiecek commented 8 years ago

It will be a good solution and I saw it in your proposition but this was not working, the .libPaths was not changing library to a specified folder (al least not on OSX)

2016-03-05 18:37 GMT+01:00 Marcin Kosiński notifications@github.com:

By now restoreLibs downgrades artifacts in the first library of .libPaths() . Wouldn't it be a good idea to install old packages in a new directory, that would not change the state of packages used by user on a daily routine? I think additional parameter lib.loc could solve such demand or we could add additional information to manual page, that a user could manually change .libPaths() on his own.

If that's ok for you additional example to restoreLibs could look something like this

old_lib <- .libPaths() new_lib <- .libPaths(lib) restoreLibs() .libPaths(old_lib)

— Reply to this email directly or view it on GitHub https://github.com/pbiecek/archivist/issues/251#issuecomment-192697078.

pozdrawiam serdecznie, Przemysław Biecek

MarcinKosinski commented 8 years ago

Could you provide a reproducible example of that situation on OSX? Something like codes and results of

.libPaths() -> x
x
.libPaths(tempfile()) -> y
y
.libPaths(x)

I'll post that case on StackOverflow

2016-03-05 20:07 GMT+01:00 Przemysław Biecek notifications@github.com:

It will be a good solution and I saw it in your proposition but this was not working, the .libPaths was not changing library to a specified folder (al least not on OSX)

2016-03-05 18:37 GMT+01:00 Marcin Kosiński notifications@github.com:

By now restoreLibs downgrades artifacts in the first library of .libPaths() . Wouldn't it be a good idea to install old packages in a new directory, that would not change the state of packages used by user on a daily routine? I think additional parameter lib.loc could solve such demand or we could add additional information to manual page, that a user could manually change .libPaths() on his own.

If that's ok for you additional example to restoreLibs could look something like this

old_lib <- .libPaths() new_lib <- .libPaths(lib) restoreLibs() .libPaths(old_lib)

— Reply to this email directly or view it on GitHub <https://github.com/pbiecek/archivist/issues/251#issuecomment-192697078 .

pozdrawiam serdecznie, Przemysław Biecek

— Reply to this email directly or view it on GitHub https://github.com/pbiecek/archivist/issues/251#issuecomment-192708541.

pbiecek commented 8 years ago

Here it is

> (x <- .libPaths())
[1] "/Library/Frameworks/R.framework/Versions/3.2/Resources/library"
> (y <- tempfile())
[1] "/var/folders/_l/jllqh32s3llbxmtrh2431vpr0000gn/T//Rtmp4qW3nJ/filee0df790601a4"
> (z <- .libPaths(y)) 
[1] "/Library/Frameworks/R.framework/Versions/3.2/Resources/library"
> .libPaths()
[1] "/Library/Frameworks/R.framework/Versions/3.2/Resources/library"
pbiecek commented 8 years ago

Ok, but now I see where is the problem Argument of .libPaths() must be a directory this one will work.

> (y <- tempfile())
[1] "/var/folders/_l/jllqh32s3llbxmtrh2431vpr0000gn/T//Rtmp4qW3nJ/filee0df3fc1811d"
> dir.create(y)
> (z <- .libPaths(y)) 
[1] "/private/var/folders/_l/jllqh32s3llbxmtrh2431vpr0000gn/T/Rtmp4qW3nJ/filee0df3fc1811d"
[2] "/Library/Frameworks/R.framework/Versions/3.2/Resources/library"                      
> .libPaths()
[1] "/private/var/folders/_l/jllqh32s3llbxmtrh2431vpr0000gn/T/Rtmp4qW3nJ/filee0df3fc1811d"
[2] "/Library/Frameworks/R.framework/Versions/3.2/Resources/library" 
MarcinKosinski commented 8 years ago

Yes, that's the case. But I still have problems here https://github.com/pbiecek/archivist/issues/255#issuecomment-192720763

2016-03-05 21:00 GMT+01:00 Przemysław Biecek notifications@github.com:

Ok, but now I see where is the problem Argument of .libPaths() must be a directory this one will work.

(y <- tempfile()) [1] "/var/folders/_l/jllqh32s3llbxmtrh2431vpr0000gn/T//Rtmp4qW3nJ/filee0df3fc1811d" dir.create(y) (z <- .libPaths(y)) [1] "/private/var/folders/_l/jllqh32s3llbxmtrh2431vpr0000gn/T/Rtmp4qW3nJ/filee0df3fc1811d" [2] "/Library/Frameworks/R.framework/Versions/3.2/Resources/library" .libPaths() [1] "/private/var/folders/_l/jllqh32s3llbxmtrh2431vpr0000gn/T/Rtmp4qW3nJ/filee0df3fc1811d" [2] "/Library/Frameworks/R.framework/Versions/3.2/Resources/library"

— Reply to this email directly or view it on GitHub https://github.com/pbiecek/archivist/issues/251#issuecomment-192720591.