Error loading phateR

[zinagood]

Thanks for making PHATE!

I just installed miniconda3 and phate. I am able to load phate with 'import phate' command in python. I also installed phateR from CRAN with no issues. The problem arises when I attempt to load the phateR library using 'library(phateR)' command. It appears that R is looking for the python package in the wrong place. Please advise to fix this.

Thanks! Zina

Here is the error I get:

install.packages("phateR") trying URL 'https://cran.rstudio.com/bin/macosx/el-capitan/contrib/3.6/phateR_1.0.0.tgz' Content type 'application/x-gzip' length 3992559 bytes (3.8 MB)

downloaded 3.8 MB

The downloaded binary packages are in /var/folders/fn/mhn7pp8s3yj7cng14ywkzyx40000gn/T//RtmpV1LBiL/downloaded_packages

library(phateR) # PHATE Error: package or namespace load failed for ‘phateR’: .onLoad failed in loadNamespace() for 'phateR', details: call: python_config(python_version, required_module, python_versions) error: Error 1 occurred running /usr/bin/python3 In addition: Warning message: In system2(command = python, args = paste0("\"", config_script, : running command ''/usr/bin/python3' "/Library/Frameworks/R.framework/Versions/3.6/Resources/library/reticulate/config/config.py" 2>/dev/null' had status 1

[scottgigante]

Can you please fill in your system information?

Output of phate.__version__:

Please run phate.__version__ and paste the results here.

You can do this with `python -c 'import phate; print(phate.__version__)'`

Output of pd.show_versions():

Please run pd.show_versions() and paste the results here.

You can do this with `python -c 'import pandas as pd; pd.show_versions()'`

Output of sessionInfo():

Please run sessionInfo() and paste the results here.

You can do this with `R -e 'library(phateR); sessionInfo()'`

Output of reticulate::py_discover_config(required_module = "phate"):

Please run `reticulate::py_discover_config(required_module = "phate")` and paste the results here.

You can do this with `R -e 'reticulate::py_discover_config(required_module = "phate")'`

[zinagood]

Thanks Scott!

phate does not give me outputs, as I am not able to load the phateR package. Still, here you go:

phate.version Python: 1.0.2 R: Error in print(phate.version) : object 'phate.version' not found

pd.show_versions() Python: Traceback (most recent call last): File "", line 1, in NameError: name 'pd' is not defined R: Error in pd.show_versions() : could not find function "pd.show_versions"

sessionInfo() in R R version 3.6.1 (2019-07-05) Platform: x86_64-apple-darwin15.6.0 (64-bit) Running under: macOS Catalina 10.15.2

Matrix products: default BLAS: /System/Library/Frameworks/Accelerate.framework/Versions/A/Frameworks/vecLib.framework/Versions/A/libBLAS.dylib LAPACK: /Library/Frameworks/R.framework/Versions/3.6/Resources/lib/libRlapack.dylib

locale: [1] en_US.UTF-8/en_US.UTF-8/en_US.UTF-8/C/en_US.UTF-8/en_US.UTF-8

attached base packages: [1] stats graphics grDevices utils datasets methods base

other attached packages: [1] reticulate_1.13 Matrix_1.2-17 ncdfFlow_2.30.1
[4] BH_1.69.0-1 RcppArmadillo_0.9.800.1.0 flowCore_1.50.0

loaded via a namespace (and not attached): [1] tidyselect_0.2.5 purrr_0.3.3 lattice_0.20-38 pcaPP_1.9-73
[5] colorspace_1.4-1 stats4_3.6.1 rlang_0.4.1 hexbin_1.27.3
[9] pillar_1.4.2 glue_1.3.1 BiocGenerics_0.30.0 RColorBrewer_1.1-2 [13] matrixStats_0.55.0 robustbase_0.93-5 zlibbioc_1.30.0 munsell_0.5.0
[17] gtable_0.3.0 mvtnorm_1.0-11 latticeExtra_0.6-28 Biobase_2.44.0
[21] parallel_3.6.1 DEoptimR_1.0-8 Rcpp_1.0.2 KernSmooth_2.23-16 [25] corpcor_1.6.9 scales_1.0.0 graph_1.62.0 jsonlite_1.6
[29] IDPmisc_1.1.19 ggplot2_3.2.1 dplyr_0.8.3 grid_3.6.1
[33] tools_3.6.1 magrittr_1.5 lazyeval_0.2.2 tibble_2.1.3
[37] cluster_2.1.0 crayon_1.3.4 rrcov_1.4-7 pkgconfig_2.0.3
[41] MASS_7.3-51.4 flowViz_1.48.0 assertthat_0.2.1 rstudioapi_0.10
[45] R6_2.4.0 compiler_3.6.1

reticulate::py_discover_config(required_module = "phate") Error in python_config(python_version, required_module, python_versions) : Error 1 occurred running /usr/bin/python3 In addition: Warning message: In system2(command = python, args = paste0("\"", config_script, : running command ''/usr/bin/python3' "/Library/Frameworks/R.framework/Versions/3.6/Resources/library/reticulate/config/config.py" 2>/dev/null' had status 1

[scottgigante]

Hi @zinagood , the first two are to be run in python. You can do so by running these commands in a terminal.

python -c 'import phate; print(phate.__version__)'
python -c 'import pandas as pd; pd.show_versions()'

However, it appears the issue is with reticulate. Can you try running the following command in R?

library(reticulate)
py_config()

[zinagood]

Thanks Scott!

OK - here are the outputs:

(base) DN2tevk1:~ zinaida$ python -c 'import phate; print(phate.version)'

1.0.2

(base) DN2tevk1:~ zinaida$ python -c 'import pandas as pd; pd.show_versions()'

INSTALLED VERSIONS

---

commit : None

python : 3.7.4.final.0

python-bits : 64

OS : Darwin

OS-release : 19.2.0

machine : x86_64

processor : i386

byteorder : little

LC_ALL : None

LANG : en_US.UTF-8

LOCALE : en_US.UTF-8

pandas : 1.0.0

numpy : 1.18.1

pytz : 2019.3

dateutil : 2.8.1

pip : 19.3.1

setuptools : 41.4.0

Cython : None

pytest : None

hypothesis : None

sphinx : None

blosc : None

feather : None

xlsxwriter : None

lxml.etree : None

html5lib : None

pymysql : None

psycopg2 : None

jinja2 : None

IPython : None

pandas_datareader: None

bs4 : None

bottleneck : None

fastparquet : None

gcsfs : None

lxml.etree : None

matplotlib : 3.1.2

numexpr : None

odfpy : None

openpyxl : None

pandas_gbq : None

pyarrow : None

pytables : None

pytest : None

pyxlsb : None

s3fs : None

scipy : 1.4.1

sqlalchemy : None

tables : None

tabulate : None

xarray : None

xlrd : None

xlwt : None

xlsxwriter : None

numba : None

library(reticulate)

py_config()

python: /usr/bin/python

libpython: /System/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/config/libpython2.7.dylib

pythonhome: /System/Library/Frameworks/Python.framework/Versions/2.7:/System/Library/Frameworks/Python.framework/Versions/2.7

version: 2.7.16 (default, Nov 9 2019, 05:55:08) [GCC 4.2.1 Compatible Apple LLVM 11.0.0 (clang-1100.0.32.4) (-macos10.15-objc-s

numpy: /System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/numpy

numpy_version: 1.8.0

python versions found:

/usr/bin/python

/usr/bin/python3

You're right - it looks like R is looking at Python 2.7 (not 3.7). I tried running this command, but that did not help.

use_python("/usr/local/bin/python3.7")

Again, I really appreciate your time helping me to get started.

Zina

-- Zinaida Good 415-290-8944

On Thu, Jan 30, 2020 at 11:18 AM Scott Gigante notifications@github.com wrote:

Hi @zinagood https://github.com/zinagood , the first two are to be run in python. You can do so by running these commands in a terminal.

python -c 'import phate; print(phate.version)' python -c 'import pandas as pd; pd.show_versions()'

However, it appears the issue is with reticulate. Can you try running the following command in R?

library(reticulate) py_config()

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KrishnaswamyLab/phateR/issues/45?email_source=notifications&email_token=ACAN5WZMNADP3GJW7E56EPDRAMRX5A5CNFSM4KN27R6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKMF6EQ#issuecomment-580411154, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAN5W5454YPSTORELP44RDRAMRX5ANCNFSM4KN27R6A .

[scottgigante]

Hi Zina,

No problem! Happy to help. What happens if you set the environment variable RETICULATE_PYTHON?

export RETICULATE_PYTHON="/usr/local/bin/python3.7"
R -e "reticulate::py_discover_config('phate')"

[zinagood]

Thanks! The first command went well, but the second command fails. I tried ending the second command with 'python3' for good luck, but that didn't help.

(base) DN2tevk1:~ zinaida$ export RETICULATE_PYTHON="/usr/local/bin/python3.7" (base) DN2tevk1:~ zinaida$ R -e "reticulate::py_discover_config('phate')"

R version 3.6.1 (2019-07-05) -- "Action of the Toes" Copyright (C) 2019 The R Foundation for Statistical Computing Platform: x86_64-apple-darwin15.6.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY. You are welcome to redistribute it under certain conditions. Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors. Type 'contributors()' for more information and 'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or 'help.start()' for an HTML browser interface to help. Type 'q()' to quit R.

reticulate::py_discover_config('phate') Error in reticulate::py_discover_config("phate") : Python specified in RETICULATE_PYTHON (/usr/local/bin/python3.7) does not exist Execution halted

[scottgigante]

Ah, my mistake, I thought you had used /usr/local/bin/python3.7 because that was the path to your local python3 installation. You can use which python in a terminal to find out the correct path.

[zinagood]

Looks like progress!

Reticulate can see Python and phate package. The problem is that 'library(phateR)" still doesn't work in R...

(base) DN2tevk1:~ zinaida$ which python

/opt/miniconda3/bin/python

(base) DN2tevk1:~ zinaida$ export RETICULATE_PYTHON="/opt/miniconda3/bin/python"

(base) DN2tevk1:~ zinaida$ R -e "reticulate::py_discover_config('phate')"

R version 3.6.1 (2019-07-05) -- "Action of the Toes"

Copyright (C) 2019 The R Foundation for Statistical Computing

Platform: x86_64-apple-darwin15.6.0 (64-bit)

R is free software and comes with ABSOLUTELY NO WARRANTY.

You are welcome to redistribute it under certain conditions.

Type 'license()' or 'licence()' for distribution details.

Natural language support but running in an English locale

R is a collaborative project with many contributors.

Type 'contributors()' for more information and

'citation()' on how to cite R or R packages in publications.

Type 'demo()' for some demos, 'help()' for on-line help, or

'help.start()' for an HTML browser interface to help.

Type 'q()' to quit R.

reticulate::py_discover_config('phate')

python: /opt/miniconda3/bin/python

libpython: /opt/miniconda3/lib/libpython3.7m.dylib

pythonhome: /opt/miniconda3:/opt/miniconda3

version: 3.7.4 (default, Aug 13 2019, 15:17:50) [Clang 4.0.1 (tags/RELEASE_401/final)]

numpy: /Users/zinaida/.local/lib/python3.7/site-packages/numpy

numpy_version: 1.18.1

phate: /Users/zinaida/.local/lib/python3.7/site-packages/phate

NOTE: Python version was forced by RETICULATE_PYTHON

---

library(phateR)

Loading required package: Matrix

Error: package or namespace load failed for ‘phateR’:

.onLoad failed in loadNamespace() for 'phateR', details:

call: python_config(python_version, required_module, python_versions)

error: Error 1 occurred running /usr/bin/python3

In addition: Warning message:

In system2(command = python, args = paste0("\"", config_script, :

running command ''/usr/bin/python3' "/Library/Frameworks/R.framework/Versions/3.6/Resources/library/reticulate/config/config.py" 2>/dev/null' had status 1

Or if I update the Python path:

use_python("/opt/miniconda3/bin/python")

library(phateR)

Error in py_module_import(module, convert = convert) :

ImportError: No module named phate

The phate Python installation command places the package here:

/Users/zinaida/.local/lib/python3.7/site-packages/phate

How could I help R see it?

Thanks!!

-- Zinaida Good 415-290-8944

On Thu, Jan 30, 2020 at 1:32 PM Scott Gigante notifications@github.com wrote:

Ah, my mistake, I thought you had used /usr/local/bin/python3.7 because that was the path to your local python3 installation. You can use which python in a terminal to find out the correct path.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KrishnaswamyLab/phateR/issues/45?email_source=notifications&email_token=ACAN5W3MCIZKFXPNR7MF3DLRANBNDA5CNFSM4KN27R6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKMUHSI#issuecomment-580469705, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAN5W2DJYEUETXKQD3QNG3RANBNDANCNFSM4KN27R6A .

[scottgigante]

Okay, that's something. Can you please try the following?

/opt/miniconda3/bin/python -c 'import phate; print(phate.__version__)'
/usr/bin/python3 -c 'import phate; print(phate.__version__)'
which pip

I suspect PHATE is installed in the system python which is confusing reticulate.

[zinagood]

Here you go - hope it can give you clues.

(base) DN2tevk1:~ zinaida$ /opt/miniconda3/bin/python -c 'import phate; print(phate.version)'

1.0.2

(base) DN2tevk1:~ zinaida$ /usr/bin/python3 -c 'import phate; print(phate.version)'

xcrun: error: invalid active developer path (/Library/Developer/CommandLineTools), missing xcrun at: /Library/Developer/CommandLineTools/usr/bin/xcrun

(base) DN2tevk1:~ zinaida$ which pip

/opt/miniconda3/bin/pip

(base) DN2tevk1:~ zinaida$

Zinaida Good 415-290-8944

On Thu, Jan 30, 2020 at 2:17 PM Scott Gigante notifications@github.com wrote:

Okay, that's something. Can you please try the following?

/opt/miniconda3/bin/python -c 'import phate; print(phate.version)' /usr/bin/python3 -c 'import phate; print(phate.version)' which pip

I suspect PHATE is installed in the system python which is confusing reticulate.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KrishnaswamyLab/phateR/issues/45?email_source=notifications&email_token=ACAN5WYAXTOAT6IMV34NRDLRANGV7A5CNFSM4KN27R6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKMYVEQ#issuecomment-580487826, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAN5W6ENRG77ZPNSKY7P2TRANGV7ANCNFSM4KN27R6A .

[scottgigante]

Okay, so it appears something about your /usr/bin/python3 is super broken. Then, because phateR is running py_discover_config to automatically choose the right python version, reticulate is running all versions of python to see which has phate installed. This is hitting the broken python3 and crashing.

You might be able to avoid the problem by running the following in terminal:

rm -rf /Users/zinaida/.local/lib/python3.7/site-packages/phate
/opt/miniconda3/bin/pip install --upgrade phate

but ultimately you may want to troubleshoot what is going wrong with your system python. You can double check my concern by running

/usr/bin/python3 -c 'import pandas; print(pandas.__version__)'

[zinagood]

Thanks a lot for helping me figure this out Scott!

I ended up creating a virtual environment with reticulate and force the use of its packages. Now everything works and results look good.

Zina

Zinaida Good 415-290-8944

On Fri, Jan 31, 2020 at 10:54 AM Scott Gigante notifications@github.com wrote:

Okay, so it appears something about your /usr/bin/python3 is super broken. Then, because phateR is running py_discover_config to automatically choose the right python version, reticulate is running all versions of python to see which has phate installed. This is hitting the broken python3 and crashing.

You might be able to avoid the problem by running the following in terminal:

rm -rf /Users/zinaida/.local/lib/python3.7/site-packages/phate /opt/miniconda3/bin/pip install --upgrade phate

but ultimately you may want to troubleshoot what is going wrong with your system python. You can double check my concern by running

/usr/bin/python3 -c 'import pandas; print(pandas.version)'

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHub https://github.com/KrishnaswamyLab/phateR/issues/45?email_source=notifications&email_token=ACAN5W2ZMJKPS32ER6GSKALRARXXDA5CNFSM4KN27R6KYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOEKPUN6I#issuecomment-580863737, or unsubscribe https://github.com/notifications/unsubscribe-auth/ACAN5W3BP3OXOPDEKJDTMCDRARXXDANCNFSM4KN27R6A .

[scottgigante]

Great, I'm glad it's sorted. Feel free to open another issue if you run into any further problems.