search

rstudio / packrat

Packrat is a dependency management system for R
http://rstudio.github.io/packrat/
401 stars 89 forks source link

Packrat hanging upon sourcing init.R #364

Open Ncalverley opened 7 years ago

Ncalverley commented 7 years ago

This is a bit of a continuation of this issue: [https://github.com/rstudio/packrat/issues/290] This hanging issue was never really solved, the original poster simply found some kind of workaround.

I stepped through the init.R code, and I tracked the exact source of the hanging to this line of code:

return(packrat::on(print.banner = print.banner)) (Line 45 of init.R)

So I dug through the guts of the packrat::on() function, and I found the exact spot in which the hang is occurring. It's happening upon a call to the .libPaths() function. Specifically, the exact call to libPaths() which is causing the problem is happening within a function buried within the packrat::on() function. See this figure:

packrat::on() -->setPackratModeOn() 
                                 -->afterPackratModeOn()
                                                                         |
                                                                         --> setLibPaths()
                                                                                                  |
                                                                                                  --> `.libPaths(paths)` (Line 6)

So, the exact spot in which the hang is occurring is on Line 6 of the setLibPaths() function. From what I can tell, this is the spot where packrat is attempting to add the packrat library paths to R's library locations. However, for some reason, this call is causing everything to hang. I don't know exactly how long it hangs for, as I've not had the patience to sit around waiting for it to finish hanging, but it's at least 10 minutes.

Here's my R version:

R version 3.2.2 (2015-08-14) -- "Fire Safety" Copyright (C) 2015 The R Foundation for Statistical Computing Platform: x86_64-w64-mingw32/x64 (64-bit)

It's an old version, I know, but we have deliberately avoided upgrading because we have a stable set of code packages that we use for our code, and that interact well with 3.2.2.

I'm using the most-recent version of RStudio, version 1.0.136

Ncalverley commented 7 years ago

Update - packrat has now been hanging for more than 30 minutes. Still do not know the ceiling on the hang time yet.

kevinushey commented 7 years ago

It seems pretty surprising / unlikely that packrat would be hanging in a call to .libPaths(). Does this hang only occur in RStudio, or do you see it in the console as well?

Ncalverley commented 7 years ago

Looks like it occurs only in RStudio. I was able to run the entire packrat::on() function successfully (and instantaneously) in the R terminal. What does that mean?

kevinushey commented 7 years ago

Hard to say for sure, but RStudio may be hanging while attempting to index something in those library paths.

What packages are you using in this project? (What's the output of packrat:::appDependencies()?) I can see if I can reproduce the slowness myself and diagnose further.

Ncalverley commented 7 years ago

packrat:::appDependencies() [1] "Amelia" "BH" "C50" "DBI" "DEoptimR" "DMwR" "EWStools" "GAMBoost" "ISLR"
[10] "KMsurv" "KernSmooth" "MASS" "Matching" "Matrix" "MatrixModels" "NbClust" "R6" "RColorBrewer"
[19] "RCurl" "RJSONIO" "ROCR" "RRF" "Rcpp" "RcppArmadillo" "RcppEigen" "SparseM" "TH.data"
[28] "TTR" "TeachingDemos" "XLConnect" "XLConnectJars" "abind" "ada" "arm" "assertthat" "base64enc"
[37] "bitops" "boot" "brew" "caTools" "car" "caret" "caretEnsemble" "chron" "class"
[46] "cluster" "coda" "codetools" "coin" "colorspace" "combinat" "corpcor" "curl" "data.table"
[55] "devtools" "dichromat" "digest" "diptest" "doParallel" "doSNOW" "dplyr" "e1071" "earth"
[64] "eeptools" "elasticnet" "entropy" "evaluate" "fdrtool" "flexmix" "foreach" "foreign" "formatR"
[73] "fpc" "gam" "gbm" "gdata" "ggplot2" "git2r" "glmnet" "gplots" "gtable"
[82] "gtools" "hda" "hexbin" "highr" "htmltools" "httpuv" "httr" "ipred" "iterators"
[91] "itertools" "jsonlite" "kernlab" "klaR" "knitr" "kohonen" "labeling" "lars" "lattice"
[100] "lava" "lazyeval" "lcmm" "lme4" "lmtest" "magrittr" "maptools" "markdown" "mboost"
[109] "mclust" "mda" "memisc" "memoise" "mgcv" "mice" "mime" "minqa" "missForest"
[118] "modeltools" "multcomp" "munsell" "mvtnorm" "neuralnet" "nlme" "nloptr" "nnet" "nnls"
[127] "nodeHarvest" "numDeriv" "pROC" "packrat" "pamr" "party" "partykit" "pbapply" "pbkrtest"
[136] "plotmo" "plotrix" "pls" "plyr" "poLCA" "prabclus" "prodlim" "proto" "proxy"
[145] "psy" "qrnn" "quadprog" "quantmod" "quantreg" "quantregForest" "rJava" "randomForest" "reshape2"
[154] "rjson" "rmarkdown" "robustbase" "roxygen2" "rpart" "rstudioapi" "rversions" "sandwich" "scales"
[163] "scatterplot3d" "sda" "servr" "snow" "sp" "sparseLDA" "spls" "stabs" "stringi"
[172] "stringr" "strucchange" "survival" "tidyr" "trimcluster" "vcd" "whisker" "xml2" "xtable"
[181] "xts" "yaml" "zoo"

Ncalverley commented 7 years ago

Here's the same list in an excel file Book1.xlsx .

Ncalverley commented 7 years ago

While we investigate this hanging issue, I'm investigating a temporary solution that essentially turns off all packrat functionality except for setting the packrat library as the default library. Basically what I'm doing is to comment out the call to source("packrat/init.R") within the .RProfile file.

Then, I manually add the packrat library as the default library:

.libPaths("C:/my_project/packrat_library", .libPaths())

Now, I know that this completely turns off all the package tracking functionalities. But, working under the assumption that the packages (and their versions) being used in the project never change, would this work to initialize all of the packages in the packrat library for use in the project?

Ncalverley commented 7 years ago

Ok, I've found a slightly different but highly related issue.

I tried completely deleting the project from my computer, and then re-cloning the project from the remote repo back onto my computer. All of the libraries installed properly, the hanging went away (with the exception of a roughly 30 second hang upon starting up the project), and everything worked great.

But then, I created a new branch and changed to it, with this new branch essentially being a carbon copy of the master branch, as I had not yet made any changes to it. Then I closed RStudio, loaded it back up again (with the new branch still open), and lo-and-behold, another 30+ minute hang.

No idea what's going on here, but packrat seems to fail pretty hard with large projects utilizing a large number of libraries. Has any progress been made with this issue?