search

rstudio / reticulate

R Interface to Python
https://rstudio.github.io/reticulate
Apache License 2.0
1.68k stars 328 forks source link

virtualenv_install problems with python 3 venv or virtualenv #393

Open colearendt opened 5 years ago

colearendt commented 5 years ago

Related to #370 , which added some great improvements for use_virtualenv on Python 3 environments.

Also related to #363 , which tracks python3 support for virtualenv_create.

It seems that virtualenv_install still does not work great with Python 3 environments (i.e. relative paths). Example with a virtualenv:

system("virtualenv -p /usr/bin/python3 py3-virtualenv")
fs::dir_ls()
#> py3-virtualenv         reprex_reprex.R        reprex_reprex.spin.R   
#> reprex_reprex.spin.Rmd

reticulate::use_virtualenv("./py3-virtualenv", required = TRUE)
reticulate::py_config()
#> python:         ./py3-virtualenv/bin/python
#> libpython:      /usr/lib/python3.4/config-3.4m-x86_64-linux-gnu/libpython3.4.so
#> pythonhome:     /usr:/usr
#> virtualenv:     ./py3-virtualenv/bin/activate_this.py
#> version:        3.4.3 (default, Nov 12 2018, 22:25:49)  [GCC 4.8.4]
#> numpy:           [NOT FOUND]
#> 
#> NOTE: Python version was forced by use_python function

reticulate::virtualenv_install("./py3-virtualenv", "numpy")
#> virtualenv: ~/.virtualenvs/./py3-virtualenv
#> Warning in system(cmd, intern = TRUE, ignore.stderr = TRUE): running
#> command '/bin/bash -c "source '/home/cole/.virtualenvs/./py3-virtualenv/
#> bin/activate' && '/home/cole/.virtualenvs/./py3-virtualenv/bin/pip' --
#> version" 2>/dev/null' had status 1
#> Warning in value[[3L]](cond): Error occurred checking pip version:
#> subscript out of bounds
#> Upgrading pip ...
#> Error: Error 1 occurred installing packages

Created on 2018-11-28 by the reprex package (v0.2.1)

Session info ``` r devtools::session_info() #> Session info ------------------------------------------------------------- #> setting value #> version R version 3.5.1 (2018-07-02) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> tz Etc/UTC #> date 2018-11-28 #> Packages ----------------------------------------------------------------- #> package * version date source #> backports 1.1.2 2017-12-13 RSPM (R 3.5.0) #> base * 3.5.1 2018-07-10 local #> compiler 3.5.1 2018-07-10 local #> crayon 1.3.4 2017-09-16 RSPM (R 3.5.0) #> datasets * 3.5.1 2018-07-10 local #> devtools 1.13.5 2018-02-18 RSPM (R 3.5.0) #> digest 0.6.15 2018-01-28 RSPM (R 3.5.0) #> evaluate 0.10.1 2017-06-24 RSPM (R 3.5.0) #> fs 1.2.6 2018-08-23 RSPM (R 3.5.1) #> graphics * 3.5.1 2018-07-10 local #> grDevices * 3.5.1 2018-07-10 local #> grid 3.5.1 2018-07-10 local #> htmltools 0.3.6 2017-04-28 RSPM (R 3.5.0) #> jsonlite 1.5 2017-06-01 RSPM (R 3.5.0) #> knitr 1.20 2018-02-20 RSPM (R 3.5.0) #> lattice 0.20-35 2017-03-25 RSPM (R 3.5.0) #> magrittr 1.5 2014-11-22 RSPM (R 3.5.0) #> Matrix 1.2-14 2018-04-09 RSPM (R 3.5.0) #> memoise 1.1.0 2017-04-21 RSPM (R 3.5.0) #> methods * 3.5.1 2018-07-10 local #> Rcpp 1.0.0 2018-11-07 cran (@1.0.0) #> reticulate 1.10.0.9003 2018-11-23 Github (rstudio/reticulate@4fa33ff) #> rmarkdown 1.9 2018-03-01 RSPM (R 3.5.0) #> rprojroot 1.3-2 2018-01-03 RSPM (R 3.5.0) #> stats * 3.5.1 2018-07-10 local #> stringi 1.2.2 2018-05-02 RSPM (R 3.5.0) #> stringr 1.3.1 2018-05-10 RSPM (R 3.5.0) #> tools 3.5.1 2018-07-10 local #> utils * 3.5.1 2018-07-10 local #> withr 2.1.2 2018-03-15 RSPM (R 3.5.0) #> yaml 2.1.19 2018-05-01 RSPM (R 3.5.0) ```

And with a venv:

system("python3 -m venv py3-venv")
fs::dir_ls()
#> py3-venv               reprex_reprex.R        reprex_reprex.spin.R   
#> reprex_reprex.spin.Rmd

reticulate::use_virtualenv("./py3-venv", required = TRUE)
reticulate::py_config()
#> python:         ./py3-venv/bin/python
#> libpython:      /usr/lib/python3.4/config-3.4m-x86_64-linux-gnu/libpython3.4.so
#> pythonhome:     /usr:/usr
#> version:        3.4.3 (default, Nov 12 2018, 22:25:49)  [GCC 4.8.4]
#> numpy:           [NOT FOUND]
#> 
#> NOTE: Python version was forced by use_python function

reticulate::virtualenv_install("./py3-venv", "numpy")
#> virtualenv: ~/.virtualenvs/./py3-venv
#> Warning in system(cmd, intern = TRUE, ignore.stderr = TRUE): running
#> command '/bin/bash -c "source '/home/cole/.virtualenvs/./py3-venv/bin/
#> activate' && '/home/cole/.virtualenvs/./py3-venv/bin/pip' --version" 2>/
#> dev/null' had status 1
#> Warning in value[[3L]](cond): Error occurred checking pip version:
#> subscript out of bounds
#> Upgrading pip ...
#> Error: Error 1 occurred installing packages

Created on 2018-11-28 by the reprex package (v0.2.1)

Session info ``` r devtools::session_info() #> Session info ------------------------------------------------------------- #> setting value #> version R version 3.5.1 (2018-07-02) #> system x86_64, linux-gnu #> ui X11 #> language (EN) #> collate en_US.UTF-8 #> tz Etc/UTC #> date 2018-11-28 #> Packages ----------------------------------------------------------------- #> package * version date source #> backports 1.1.2 2017-12-13 RSPM (R 3.5.0) #> base * 3.5.1 2018-07-10 local #> compiler 3.5.1 2018-07-10 local #> crayon 1.3.4 2017-09-16 RSPM (R 3.5.0) #> datasets * 3.5.1 2018-07-10 local #> devtools 1.13.5 2018-02-18 RSPM (R 3.5.0) #> digest 0.6.15 2018-01-28 RSPM (R 3.5.0) #> evaluate 0.10.1 2017-06-24 RSPM (R 3.5.0) #> fs 1.2.6 2018-08-23 RSPM (R 3.5.1) #> graphics * 3.5.1 2018-07-10 local #> grDevices * 3.5.1 2018-07-10 local #> grid 3.5.1 2018-07-10 local #> htmltools 0.3.6 2017-04-28 RSPM (R 3.5.0) #> jsonlite 1.5 2017-06-01 RSPM (R 3.5.0) #> knitr 1.20 2018-02-20 RSPM (R 3.5.0) #> lattice 0.20-35 2017-03-25 RSPM (R 3.5.0) #> magrittr 1.5 2014-11-22 RSPM (R 3.5.0) #> Matrix 1.2-14 2018-04-09 RSPM (R 3.5.0) #> memoise 1.1.0 2017-04-21 RSPM (R 3.5.0) #> methods * 3.5.1 2018-07-10 local #> Rcpp 1.0.0 2018-11-07 cran (@1.0.0) #> reticulate 1.10.0.9003 2018-11-23 Github (rstudio/reticulate@4fa33ff) #> rmarkdown 1.9 2018-03-01 RSPM (R 3.5.0) #> rprojroot 1.3-2 2018-01-03 RSPM (R 3.5.0) #> stats * 3.5.1 2018-07-10 local #> stringi 1.2.2 2018-05-02 RSPM (R 3.5.0) #> stringr 1.3.1 2018-05-10 RSPM (R 3.5.0) #> tools 3.5.1 2018-07-10 local #> utils * 3.5.1 2018-07-10 local #> withr 2.1.2 2018-03-15 RSPM (R 3.5.0) #> yaml 2.1.19 2018-05-01 RSPM (R 3.5.0) ```

It seems that virtualenv_install is still focused on installing in the home directory.

kevinushey commented 5 years ago

@jjallaire should the virtualenv_*() set of functions work with absolute paths? Or is the intention that they're always names of environments that live within ~/.virtualenvs?

jjallaire commented 5 years ago

It seems like it would be good if they worked with absolute paths, but we'd need to dig into the implementation of each to verify that this is sane for all codepaths (i.e. no subtle/implicit dependencies on the relative paths or being subdirectories of WORKON_HOME)