Open paulstaab opened 9 years ago
I can confirm this issue - I was able to reproduce it with current stable version installed from cran (4.4).
Edit - deleting the symlink really works - yesterday i did something wrong...
The problem is here: https://github.com/hadley/packman/blob/master/old/cmd-install.r#L244-L248
Assume that we have lib dir
libsDir = '/xxx/libs'
and cache dir
cacheDir = '/xxx/cache'
and we also have library mime installed in
mimeReal = '/xxx/cache/mime'
and this library is symlinked to
mimeLinked = '/xxx/libs/mime'
so now we want to install and in one phase of the install script, it calls
is_subdir(instdir, lib)
where instdir = mimeLinked
and lib = libsDir
everything seems ok, but what the check does is that it tries to compare normalized paths of instdir/
.. and lib
, but the problem is that symlinks are expanded during normalization...
I think we cannot expect some updates from R devs, so @paulstaab's solutions seems to be the best.
I also ran into this problem, but I can't quite find the /xxx/cache/mime
in order to remove the symlink. I followed this path to /home/my_user/.config/packrat/v2/library/*
, but I don't see /cache
. Any hints?
Thanks -- I definitely agree that packrat should do what it can to move symlinks out of the way on package install, and this is something I hope we'll get to for the next packrat release.
@statwonk, you should be able to find the packrat cache directory with (as you pointed out):
packrat:::cacheLibDir()
That directory itself is where the packages will live, so you should see the names of packages within that directory.
When I look in the directory returned by packrat:::cacheLibDir()
, I don't see any symlinks, just a tree structure like this:
➜ library tree | head
.
├── arules
│ └── d4b626ba8a4659c3f847f97d898b6f4f
│ └── arules
│ ├── CITATION
│ ├── data
│ │ ├── Adult.rda
│ │ ├── AdultUCI.rda
│ │ ├── Epub.rda
│ │ ├── Groceries.rda
@kevinushey to resolve the is not a legal package name
problem, can I just rm -rf library/
?
Ah, the symlinks are in your actual packrat directory -- if you look at the contents of
packrat:::libDir()
you should see a bunch of directory symlinks to the cache folder; you want to remove the symlinks there, not the actual files in the cache.
:+1: Reproduced today with
install.packages("formatR")
while using packrat 0.4.6-1 on openSUSE 13.2 with the global cache option enabled in the packrat section of RStudio 0.99.489.
I get the same error (ERROR: ‘xxx’ is not a legal package name) trying to update boot, MASS, and spatial using packrat 0.4.7-1 from RStudio 0.99.1130 on Ubuntu 14.04.
packrat options: Automatically snapshot local changes: TRUE Use global cache for installed packages: FALSE
The error is not specific to boot, MASS, and spatial - they are just the packages for which there were updates available. I tried updating cluster and got the same error message.
I am not using global cache and when I look in packrat:::libDir() there are no symlinks, everything is a standard directory.
packrat/lib-R is full of symlinks to the relevant packages. I tried deleting the symlink to boot and updating boot. I got the same error message and the symlink was recreated. I tried deleting the packrat/lib-R directory and that was recreated along with all the symlinks in it, and I still get the error message.
All the symlinks in lib-R are owned by root and have restrictive permissions. Is that the way it should be? I tried sudo rstudio, but got the same error.
sessionInfo() R version 3.2.4 Revised (2016-03-16 r70336) Platform: x86_64-pc-linux-gnu (64-bit) Running under: Ubuntu 14.04.4 LTS
locale:
[1] LC_CTYPE=en_AU.UTF-8 LC_NUMERIC=C LC_TIME=en_AU.UTF-8
[4] LC_COLLATE=en_AU.UTF-8 LC_MONETARY=en_AU.UTF-8 LC_MESSAGES=en_AU.UTF-8
[7] LC_PAPER=en_AU.UTF-8 LC_NAME=C LC_ADDRESS=C
[10] LC_TELEPHONE=C LC_MEASUREMENT=en_AU.UTF-8 LC_IDENTIFICATION=C
attached base packages: [1] stats graphics grDevices utils datasets methods base
other attached packages: [1] RMySQL_0.10.8 DBI_0.3.1
loaded via a namespace (and not attached): [1] Rcpp_0.12.4 packrat_0.4.7-1 digest_0.6.9 dplyr_0.4.3 assertthat_0.1 [6] R6_2.1.2 formatR_1.3 magrittr_1.5 stringi_1.0-1 rmarkdown_0.9.5 [11] tools_3.2.4 stringr_1.0.0 yaml_2.1.13 parallel_3.2.4 htmltools_0.3.5 [16] knitr_1.12.3
Get the same error. I've noticed the packages affected are packages that are updated when R is updated. For example, I've recently updated from 3.2.x to 3.3.x and I can't update the boot, MASS, spatial, and survival packages but other packages that don't require root privileges have updated in packrat fine.
Any word on a word around or a fix? I've tried checking for symlink files but don't have any as @rgayler has pointed out.
Thanks.
What is the output of find.package("MASS")
? If that is found in the lib-R
folder, and you're using caching, it's likely a symlink.
If so, things should be fine if you explicitly remove the offending packages and then reinstall them, e.g.
remove.packages("MASS")
install.packages("MASS")
Thanks @kevinushey that seems to have done the trick
Thanks @kevinushey https://github.com/kevinushey. I confirm that also works for me.
I also noted the following points:
The RStudio packages pane divides packages into two separate sections: Packrat Library, System Library
The packages that cause an error message when I attempt to update via the update button are all in the System Library.
When I use the update packages button for a package in the System Library the generated code tries to install it in the Packrat Library
install.packages("boot", lib="/home/ross/CA/projects/active/project_name/subproject_name/packrat/lib-R") trying URL ' https://cran.revolutionanalytics.com/src/contrib/boot_1.3-18.tar.gz'
Content type 'unknown' length 231831 bytes (226 KB)
downloaded 226 KB
ERROR: ‘boot’ is not a legal package name Warning in install.packages : installation of package ‘boot’ had non-zero exit status
install.packages("some_system_library_package") updates the package OK without needing to remove it first.
Could the original problem be as simple as the update button wrongly assuming that all packages are in the packrat libray when packrat is used?
However, I now see that I have the package in both libraries. I presume that this is safe and that code run in the packrat project will search the packrat library before the system library. I also presume that removing a package from the system library will make it unavailable to all non-packrat projects.
Finally, I notice that the update packages dialog is still showing the updated packages as needing update. That is, it appears to be looking at all packages in the packrat and system libraries and flagging if any of them need updating. Shouldn't the behaviour in a packrat project be that if a package exists in the packrat and system libraries then the update dialog should only look at the package in the packrat library?
Ross
On 19 July 2016 at 07:01, Phil Mike Jones notifications@github.com wrote:
Thanks @kevinushey https://github.com/kevinushey that seems to have done the trick
— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/rstudio/packrat/issues/228#issuecomment-233457681, or mute the thread https://github.com/notifications/unsubscribe-auth/AFKJG23ZD16KGbfEtJ4bp2yq5nGjQYAVks5qW-nHgaJpZM4FEJD7 .
This has been discussed and documented as a bug in R:
which is fixed in R-devel:
I'm not sure, but I'm wondering if I'm seeing the same problem with different symptoms. I'm getting the following error when I open a Packrat-enabled project in Rstudio:
Error in ensurePackageSymlink(source, target) :
Target '/home/me/Desktop/my-project/packrat/lib-R/boot' already exists and is not a symlink
I'm confused about the actual cause and the appropriate workaround.
There are other discussions relating to Packrat and symlinking here on GitHub. For instance https://github.com/rstudio/packrat/issues/276 and https://github.com/rstudio/packrat/issues/323. There's also a discussion of this error on StackOverflow. There, the workaround suggested is deleting the directory <project>/packrat/lib-R
.
The bit that confuses me, if I'm reading Bugzilla correctly, is that this is apparently fixed in R 3.2.x
or 3.3.x
. However, I'm running into the issue with R 3.4.1
. Here's my sessionInfo:
R version 3.4.1 (2017-06-30)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Fedora 26 (Workstation Edition)
Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets
[6] methods base
loaded via a namespace (and not attached):
[1] compiler_3.4.1 tools_3.4.1 packrat_0.4.8-1
Can someone offer me simple guidance? I use R infrequently, and I'm just hoping to follow best practices & standard conventions.
cc/ @jimhester
@b-long, I think this particular issue should be resolved in the development version of Packrat. Can you try installing it with:
devtools::install_github("rstudio/packrat")
and let me know if you can still reproduce the issue?
Hi guys,
Please DO NOT MAKE ANY SYMLINKS for doing make some packrat on Linux environment. I can always reproduce with @b-long, that's the reason why @b-long see that. Must to fix it.
Mainly, I use packrat on Windows and Ubuntu both. On the Windows, the boot seems not symlinked, but on Ubuntu, that always symlinked. So that I always experience with @b-long, that's too disturbing do work.
One thing more. I operate packrat with the Dropbox. All projects were syncing with Dropbox among Heterogeneous operating system. But Dropbox do not allow any symlinks. That's my pain point nowadays.
Seongho
Hi @kevinushey, thanks for the suggestion 👍 This seemed to resolve the issue on my Fedora box, but not my Mac.
The change that I made is as follows:
packrat/src/
0.4.8.1
, per your instructions (see diff below).I committed the above changes, then git pull
-ed to my Mac. I confirmed that packrat/src/
was removed. Next, I open Rstudio, and the output below is given:
R version 3.4.1 (2017-06-30) -- "Single Candle"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin15.6.0 (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
Error in ensurePackageSymlink(source, target) :
Target '/home/me/Desktop/my-project/packrat/lib-R/boot' already exists and is not a symlink
The sessionInfo()
on my Mac:
R version 3.4.1 (2017-06-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.5
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
other attached packages:
[1] packrat_0.4.8-1
loaded via a namespace (and not attached):
[1] compiler_3.4.1 tools_3.4.1
Is this expected behavior?
Packrat version being used:
diff --git a/packrat/packrat.lock b/packrat/packrat.lock
index 45fa9fb..102e496 100644
--- a/packrat/packrat.lock
+++ b/packrat/packrat.lock
@@ -1,5 +1,5 @@
PackratFormat: 1.4
-PackratVersion: 0.4.8.1
+PackratVersion: 0.4.8.22
RVersion: 3.4.1
Repos: CRAN=https://cran.rstudio.com/
@@ -214,9 +214,13 @@ Version: 0.9.6
Hash: 5f4711e142a44655dfea4d64fcf2f641
Package: packrat
-Source: CRAN
-Version: 0.4.8-1
-Hash: 6ad605ba7b4b476d84be6632393f5765
+Source: github
+Version: 0.4.8-22
+Hash: a1258ff4ad8716b9b1b38fdd173cf07f
+GithubRepo: packrat
+GithubUsername: rstudio
+GithubRef: master
+GithubSha1: 5091f2ca7f302d33e18a2df50b95c6b4bbf81f47
Package: pkgbuild
Source: github
@b-long: I'll try to get that fixed up; in the interim you should be able to resolve this by manually deleting the lib-R
folder. (Packrat will automatically re-generate it next time you activate packrat in that project)
I just pushed an update to Packrat that should automatically handle this case -- if you have a moment, can you try downloading packrat once more and trying again?
Hi @kevinushey, sorry for the slow reply. I've worked through the packrat
update and capture some results. See below.
First, on my Fedora machine, I updated Packrat. Here's the change:
diff --git a/packrat/packrat.lock b/packrat/packrat.lock
index 102e496..5fe5a71 100644
--- a/packrat/packrat.lock
+++ b/packrat/packrat.lock
@@ -1,5 +1,5 @@
PackratFormat: 1.4
-PackratVersion: 0.4.8.22
+PackratVersion: 0.4.8.24
RVersion: 3.4.1
Repos: CRAN=https://cran.rstudio.com/
@@ -215,12 +215,12 @@ Hash: 5f4711e142a44655dfea4d64fcf2f641
Package: packrat
Source: github
-Version: 0.4.8-22
-Hash: a1258ff4ad8716b9b1b38fdd173cf07f
+Version: 0.4.8-24
+Hash: 955a7cb8310e60306ad6201791f26f2c
GithubRepo: packrat
GithubUsername: rstudio
GithubRef: master
-GithubSha1: 5091f2ca7f302d33e18a2df50b95c6b4bbf81f47
+GithubSha1: 9492e7482bacf6421bd6303314534c085c3b744d
Package: pkgbuild
Source: github
I forgot to capture sessionInfo
before I started, sorry about that. In any case, here's the sessionInfo
after updating packrat on Fedora:
> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-redhat-linux-gnu (64-bit)
Running under: Fedora 26 (Workstation Edition)
Matrix products: default
BLAS/LAPACK: /usr/lib64/R/lib/libRblas.so
locale:
[1] LC_CTYPE=en_US.UTF-8 LC_NUMERIC=C
[3] LC_TIME=en_US.UTF-8 LC_COLLATE=en_US.UTF-8
[5] LC_MONETARY=en_US.UTF-8 LC_MESSAGES=en_US.UTF-8
[7] LC_PAPER=en_US.UTF-8 LC_NAME=C
[9] LC_ADDRESS=C LC_TELEPHONE=C
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] httr_1.2.1 compiler_3.4.1 R6_2.2.2 tools_3.4.1
[5] withr_2.0.0 curl_2.8.1 memoise_1.1.0 git2r_0.19.0
[9] digest_0.6.12 packrat_0.4.8-24 devtools_1.13.2
I committed the packrat update, pushed the changes to my git server and pulled down to my Mac. Upon opening Rstudio on my Mac, here's the output:
R version 3.4.1 (2017-06-30) -- "Single Candle"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin15.6.0 (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
Error in ensurePackageSymlink(source, target) :
Target '/home/me/Desktop/my-project/packrat/lib-R/boot' already exists and is not a symlink
As you can see, I'm still getting "Error in ensurePackageSymlink" . This time, I really think it is expected behavior, since this git working copy already has the following packrat
subdirectories:
ls -1 /home/me/Desktop/my-project/packrat/
bundles
init.R
lib
lib-R
lib-ext
packrat.lock
packrat.opts
Next, I invoke sessionInfo()
:
> sessionInfo()
R version 3.4.1 (2017-06-30)
Platform: x86_64-apple-darwin15.6.0 (64-bit)
Running under: macOS Sierra 10.12.5
Matrix products: default
BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib
LAPACK: /Library/Frameworks/R.framework/Versions/3.4/Resources/lib/libRlapack.dylib
locale:
[1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8
attached base packages:
[1] stats graphics grDevices utils datasets methods base
loaded via a namespace (and not attached):
[1] compiler_3.4.1 tools_3.4.1 packrat_0.4.8-1
It's interesting to me that for some reason Packrat is even older. Perhaps this is coming from a System library path? Again, I'll note that I'm somewhat of an R novice.
For the sake of a pristine working directory, I moved this copy out of the way and re-cloned the git repository. Note, all directories mentioned earlier aren't checked into my git repo, they were just part of that working environment. As you can see, in my freshly cloned git repo, the packrat
directory has the following subdirectories:
$ ls -1 my-project/packrat/
init.R
packrat.lock
packrat.opts
Next, I re-open Rstudio. No errors this time! So, I think what I ought to do next is actually .gitignore
the difference of packrat subdirectories & files. That is, add these to my .gitignore
file:
packrat/bundles/
packrat/lib/
packrat/lib-R/
packrat/lib-ext/
packrat/src/
Here's the (clean) output opening my Rstudio project on my Mac:
R version 3.4.1 (2017-06-30) -- "Single Candle"
Copyright (C) 2017 The R Foundation for Statistical Computing
Platform: x86_64-apple-darwin15.6.0 (64-bit)
R is free software and comes with ABSOLUTELY NO WARRANTY.
You are welcome to redistribute it under certain conditions.
Type 'license()' or 'licence()' for distribution details.
Natural language support but running in an English locale
R is a collaborative project with many contributors.
Type 'contributors()' for more information and
'citation()' on how to cite R or R packages in publications.
Type 'demo()' for some demos, 'help()' for on-line help, or
'help.start()' for an HTML browser interface to help.
Type 'q()' to quit R.
Packrat is not installed in the local library -- attempting to bootstrap an installation...
> No source tarball of packrat available locally
> Using user-library packrat (0.4.8.1) to bootstrap this project
Installing BH (1.62.0-1) ...
OK (downloaded binary)
Installing MASS (7.3-47) ...
OK (downloaded binary)
Installing R6 (2.2.2) ...
OK (downloaded binary)
Installing RColorBrewer (1.1-2) ...
OK (downloaded binary)
Installing Rcpp (0.12.12) ...
OK (downloaded binary)
Installing assertthat (0.2.0) ...
OK (downloaded binary)
Installing backports (1.1.0) ...
OK (downloaded binary)
Installing bitops (1.0-6) ...
OK (downloaded binary)
Installing brew (1.0-6) ...
OK (downloaded binary)
Installing colorspace (1.3-2) ...
OK (downloaded binary)
Installing commonmark (1.2) ...
OK (downloaded binary)
Installing crayon (1.3.2) ...
OK (downloaded binary)
Installing curl (2.8.1) ...
OK (downloaded binary)
Installing dichromat (2.0-0) ...
OK (downloaded binary)
Installing digest (0.6.12) ...
OK (downloaded binary)
Installing git2r (0.19.0) ...
OK (downloaded binary)
Installing gtable (0.2.0) ...
OK (downloaded binary)
Installing jpeg (0.1-8) ...
OK (downloaded binary)
Installing jsonlite (1.5) ...
OK (downloaded binary)
Installing labeling (0.3) ...
OK (downloaded binary)
Installing lazyeval (0.2.0) ...
OK (downloaded binary)
Installing magrittr (1.5) ...
OK (downloaded binary)
Installing maps (3.2.0) ...
OK (downloaded binary)
Installing mime (0.5) ...
OK (downloaded binary)
Installing openssl (0.9.6) ...
OK (downloaded binary)
Installing packrat (0.4.8-24) ...
OK (built source)
Installing png (0.1-7) ...
OK (downloaded binary)
Installing praise (1.0.0) ...
OK (downloaded binary)
Installing proto (1.0.0) ...
OK (downloaded binary)
Installing rjson (0.2.15) ...
OK (downloaded binary)
Installing rlang (0.1.1.9000) ...
OK (built source)
Installing rstudioapi (0.6) ...
OK (downloaded binary)
Installing sp (1.2-5) ...
OK (downloaded binary)
Installing stringi (1.1.5) ...
OK (downloaded binary)
Installing whisker (0.3-2) ...
OK (downloaded binary)
Installing withr (2.0.0) ...
OK (downloaded binary)
Installing plyr (1.8.4) ...
OK (downloaded binary)
Installing xml2 (1.1.1) ...
OK (downloaded binary)
Installing rprojroot (1.2-10) ...
OK (built source)
Installing munsell (0.4.3) ...
OK (downloaded binary)
Installing debugme (1.0.2) ...
OK (downloaded binary)
Installing memoise (1.1.0) ...
OK (downloaded binary)
Installing mapproj (1.2-5) ...
OK (downloaded binary)
Installing httr (1.2.1) ...
OK (downloaded binary)
Installing RgoogleMaps (1.4.1) ...
OK (downloaded binary)
Installing testthat (1.0.2) ...
OK (downloaded binary)
Installing tibble (1.3.3) ...
OK (downloaded binary)
Installing geosphere (1.5-5) ...
OK (downloaded binary)
Installing stringr (1.2.0) ...
OK (downloaded binary)
Installing desc (1.1.0) ...
OK (built source)
Installing here (0.1) ...
OK (downloaded binary)
Installing scales (0.4.1) ...
OK (downloaded binary)
Installing processx (2.0.1) ...
OK (built source)
Installing devtools (1.13.2) ...
OK (built source)
Installing reshape2 (1.4.2) ...
OK (downloaded binary)
Installing roxygen2 (6.0.1) ...
OK (downloaded binary)
Installing callr (1.0.0.9000) ...
OK (built source)
Installing ggplot2 (2.2.1) ...
OK (downloaded binary)
Installing pkgbuild (0.0.0.9000) ...
OK (built source)
Installing ggmap (2.6.1) ...
OK (downloaded binary)
Installing pkgload (0.0.0.9000) ...
OK (built source)
Packrat mode on. Using library in directory:
- "/home/me/Desktop/my-project/packrat/lib"
The package I'm building only uses the following dependencies: here
, utils
, jsonlite
, ggmap
, utils
. I'm totally unsure at this point if the package management model of packrat is expected to pull in things that are transitive dependencies in this way. If so, great, but the output is confusing. If not, then I think I've used packrat::snapshot()
incorrectly somehow.
One additional note that I'll add -- I will try to open source the package I'm working on as quickly as possible. Perhaps it could serve as a decent case study for those new to R or for your work in building Packrat.
Thanks for all your contributions & help! 👍
When I try to overwrite a symlinked package when using the cache option, I get
It seems this happens for all front-ends of
R CMD INSTALL
(I have testedR CMD INSTALL
itself,install.packages
,update.packages
,packrat::install
andpackrat::install_github
).My best guess is that R tries and fails to overwrite the symlink with a directory. If I manually delete the symlink, everything works. I'm not sure if you can solve this for
R CMD INSTALL
, but the packrat commands could remove the symlink before installing a new version of an existing package.