search

Sage-Bionetworks / synapser

An R package providing programmatic access to Synapse
Apache License 2.0
32 stars 21 forks source link

Error: ModuleNotFoundError: No module named 'synapseclient' #338

Closed timothytsai closed 1 month ago

timothytsai commented 1 month ago

Operating system

Ubuntu 22.04.4 LTS

Description of the problem

When trying to install synapser in Posit Workbench RStudio environment via > install.packages("synapser", repos = c("http://ran.synapse.org", "http://cran.fhcrc.org")) with reticulate virtualenv configured (Python 3.11.5), I am getting an error Error: ModuleNotFoundError: No module named 'synapseclient'.

Tried configuring the interpreter via the Workbench project/global options per https://docs.google.com/document/d/1-JbgCQxJFNVS7Brs8cFoJwE6o9wtRY2EvRVIzwg5bwA/edit#heading=h.hxbekpfx1kt9, deleting the virtual env via rm -r.

Made sure reticulate is 1.28 and Python/R versions are within requirements.

Expected behavior

Doing $ ls /home/{username}/.virtualenvs/r-reticulate/lib/python3.11/site-packages/synapseclient seems to show that synapseclient is downloaded via pip in to the virtualenv. Expect the installer to see this and proceed.

Actual behavior

> install.packages("synapser", repos = c("http://ran.synapse.org", "http://cran.fhcrc.org"))
Installing package into ‘/home/{username}/R/x86_64-pc-linux-gnu-library/4.3’
(as ‘lib’ is unspecified)
trying URL 'http://ran.synapse.org/src/contrib/synapser_2.0.0.200.tar.gz'
Content type 'application/x-tar' length 6043473 bytes (5.8 MB)
==================================================
downloaded 5.8 MB

* installing *source* package ‘synapser’ ...
** using staged installation
[1] "*** Using Python Configuration:"
python:         /opt/python/3.11.5/bin/python
libpython:      /opt/python/3.11.5/lib/libpython3.11.so
pythonhome:     /opt/python/3.11.5:/opt/python/3.11.5
version:        3.11.5 (main, Oct  5 2023, 22:57:26) [GCC 11.4.0]
numpy:           [NOT FOUND]

NOTE: Python version was forced by RETICULATE_PYTHON
Using virtual environment '~/.virtualenvs/r-reticulate' ...
+ '/home/{username}/.virtualenvs/r-reticulate/bin/python' -m pip install --upgrade --no-user 'requests<3' 'pandas~=2.0.0' 'pysftp' 'jinja2' 'markupsafe'
Collecting requests<3
  Using cached requests-2.32.3-py3-none-any.whl.metadata (4.6 kB)
Collecting pandas~=2.0.0
  Using cached pandas-2.0.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (18 kB)
Collecting pysftp
  Using cached pysftp-0.2.9-py3-none-any.whl
Collecting jinja2
  Using cached jinja2-3.1.4-py3-none-any.whl.metadata (2.6 kB)
Collecting markupsafe
  Using cached MarkupSafe-2.1.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (3.0 kB)
Collecting charset-normalizer<4,>=2 (from requests<3)
  Using cached charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (33 kB)
Collecting idna<4,>=2.5 (from requests<3)
  Using cached idna-3.7-py3-none-any.whl.metadata (9.9 kB)
Collecting urllib3<3,>=1.21.1 (from requests<3)
  Using cached urllib3-2.2.2-py3-none-any.whl.metadata (6.4 kB)
Collecting certifi>=2017.4.17 (from requests<3)
  Using cached certifi-2024.7.4-py3-none-any.whl.metadata (2.2 kB)
Collecting python-dateutil>=2.8.2 (from pandas~=2.0.0)
  Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl.metadata (8.4 kB)
Collecting pytz>=2020.1 (from pandas~=2.0.0)
  Using cached pytz-2024.1-py2.py3-none-any.whl.metadata (22 kB)
Collecting tzdata>=2022.1 (from pandas~=2.0.0)
  Using cached tzdata-2024.1-py2.py3-none-any.whl.metadata (1.4 kB)
Requirement already satisfied: numpy>=1.21.0 in /home/{username}/.virtualenvs/r-reticulate/lib/python3.11/site-packages (from pandas~=2.0.0) (2.0.1)
Collecting paramiko>=1.17 (from pysftp)
  Using cached paramiko-3.4.0-py3-none-any.whl.metadata (4.4 kB)
Collecting bcrypt>=3.2 (from paramiko>=1.17->pysftp)
  Using cached bcrypt-4.2.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (9.6 kB)
Collecting cryptography>=3.3 (from paramiko>=1.17->pysftp)
  Using cached cryptography-43.0.0-cp39-abi3-manylinux_2_28_x86_64.whl.metadata (5.4 kB)
Collecting pynacl>=1.5 (from paramiko>=1.17->pysftp)
  Using cached PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl.metadata (8.6 kB)
Collecting six>=1.5 (from python-dateutil>=2.8.2->pandas~=2.0.0)
  Using cached six-1.16.0-py2.py3-none-any.whl.metadata (1.8 kB)
Collecting cffi>=1.12 (from cryptography>=3.3->paramiko>=1.17->pysftp)
  Using cached cffi-1.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (1.5 kB)
Collecting pycparser (from cffi>=1.12->cryptography>=3.3->paramiko>=1.17->pysftp)
  Using cached pycparser-2.22-py3-none-any.whl.metadata (943 bytes)
Using cached requests-2.32.3-py3-none-any.whl (64 kB)
Using cached pandas-2.0.3-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (12.2 MB)
Using cached jinja2-3.1.4-py3-none-any.whl (133 kB)
Using cached MarkupSafe-2.1.5-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (28 kB)
Using cached certifi-2024.7.4-py3-none-any.whl (162 kB)
Using cached charset_normalizer-3.3.2-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (140 kB)
Using cached idna-3.7-py3-none-any.whl (66 kB)
Using cached paramiko-3.4.0-py3-none-any.whl (225 kB)
Using cached python_dateutil-2.9.0.post0-py2.py3-none-any.whl (229 kB)
Using cached pytz-2024.1-py2.py3-none-any.whl (505 kB)
Using cached tzdata-2024.1-py2.py3-none-any.whl (345 kB)
Using cached urllib3-2.2.2-py3-none-any.whl (121 kB)
Using cached bcrypt-4.2.0-cp39-abi3-manylinux_2_28_x86_64.whl (273 kB)
Using cached cryptography-43.0.0-cp39-abi3-manylinux_2_28_x86_64.whl (4.0 MB)
Using cached PyNaCl-1.5.0-cp36-abi3-manylinux_2_17_x86_64.manylinux2014_x86_64.manylinux_2_24_x86_64.whl (856 kB)
Using cached six-1.16.0-py2.py3-none-any.whl (11 kB)
Using cached cffi-1.16.0-cp311-cp311-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (464 kB)
Using cached pycparser-2.22-py3-none-any.whl (117 kB)
Installing collected packages: pytz, urllib3, tzdata, six, pycparser, markupsafe, idna, charset-normalizer, certifi, bcrypt, requests, python-dateutil, jinja2, cffi, pynacl, pandas, cryptography, paramiko, pysftp
Successfully installed bcrypt-4.2.0 certifi-2024.7.4 cffi-1.16.0 charset-normalizer-3.3.2 cryptography-43.0.0 idna-3.7 jinja2-3.1.4 markupsafe-2.1.5 pandas-2.0.3 paramiko-3.4.0 pycparser-2.22 pynacl-1.5.0 pysftp-0.2.9 python-dateutil-2.9.0.post0 pytz-2024.1 requests-2.32.3 six-1.16.0 tzdata-2024.1 urllib3-2.2.2
Using virtual environment '~/.virtualenvs/r-reticulate' ...
+ '/home/{username}/.virtualenvs/r-reticulate/bin/python' -m pip install --upgrade --no-user 'synapseclient==4.0.0'
Collecting synapseclient==4.0.0
  Using cached synapseclient-4.0.0-py3-none-any.whl.metadata (13 kB)
Requirement already satisfied: requests<3.0,>=2.22.0 in /home/{username}/.virtualenvs/r-reticulate/lib/python3.11/site-packages (from synapseclient==4.0.0) (2.32.3)
Collecting urllib3<2,>=1.26.18 (from synapseclient==4.0.0)
  Using cached urllib3-1.26.19-py2.py3-none-any.whl.metadata (49 kB)
Collecting deprecated<2.0,>=1.2.4 (from synapseclient==4.0.0)
  Using cached Deprecated-1.2.14-py2.py3-none-any.whl.metadata (5.4 kB)
Collecting opentelemetry-api~=1.21.0 (from synapseclient==4.0.0)
  Using cached opentelemetry_api-1.21.0-py3-none-any.whl.metadata (1.4 kB)
Collecting opentelemetry-sdk~=1.21.0 (from synapseclient==4.0.0)
  Using cached opentelemetry_sdk-1.21.0-py3-none-any.whl.metadata (1.5 kB)
Collecting opentelemetry-exporter-otlp-proto-http~=1.21.0 (from synapseclient==4.0.0)
  Using cached opentelemetry_exporter_otlp_proto_http-1.21.0-py3-none-any.whl.metadata (2.4 kB)
Collecting wrapt<2,>=1.10 (from deprecated<2.0,>=1.2.4->synapseclient==4.0.0)
  Using cached wrapt-1.16.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl.metadata (6.6 kB)
Collecting importlib-metadata<7.0,>=6.0 (from opentelemetry-api~=1.21.0->synapseclient==4.0.0)
  Using cached importlib_metadata-6.11.0-py3-none-any.whl.metadata (4.9 kB)
Collecting backoff<3.0.0,>=1.10.0 (from opentelemetry-exporter-otlp-proto-http~=1.21.0->synapseclient==4.0.0)
  Using cached backoff-2.2.1-py3-none-any.whl.metadata (14 kB)
Collecting googleapis-common-protos~=1.52 (from opentelemetry-exporter-otlp-proto-http~=1.21.0->synapseclient==4.0.0)
  Using cached googleapis_common_protos-1.63.2-py2.py3-none-any.whl.metadata (1.5 kB)
Collecting opentelemetry-exporter-otlp-proto-common==1.21.0 (from opentelemetry-exporter-otlp-proto-http~=1.21.0->synapseclient==4.0.0)
  Using cached opentelemetry_exporter_otlp_proto_common-1.21.0-py3-none-any.whl.metadata (1.8 kB)
Collecting opentelemetry-proto==1.21.0 (from opentelemetry-exporter-otlp-proto-http~=1.21.0->synapseclient==4.0.0)
  Using cached opentelemetry_proto-1.21.0-py3-none-any.whl.metadata (2.3 kB)
Collecting protobuf<5.0,>=3.19 (from opentelemetry-proto==1.21.0->opentelemetry-exporter-otlp-proto-http~=1.21.0->synapseclient==4.0.0)
  Using cached protobuf-4.25.4-cp37-abi3-manylinux2014_x86_64.whl.metadata (541 bytes)
Collecting opentelemetry-semantic-conventions==0.42b0 (from opentelemetry-sdk~=1.21.0->synapseclient==4.0.0)
  Using cached opentelemetry_semantic_conventions-0.42b0-py3-none-any.whl.metadata (2.3 kB)
Collecting typing-extensions>=3.7.4 (from opentelemetry-sdk~=1.21.0->synapseclient==4.0.0)
  Using cached typing_extensions-4.12.2-py3-none-any.whl.metadata (3.0 kB)
Requirement already satisfied: charset-normalizer<4,>=2 in /home/{username}/.virtualenvs/r-reticulate/lib/python3.11/site-packages (from requests<3.0,>=2.22.0->synapseclient==4.0.0) (3.3.2)
Requirement already satisfied: idna<4,>=2.5 in /home/{username}/.virtualenvs/r-reticulate/lib/python3.11/site-packages (from requests<3.0,>=2.22.0->synapseclient==4.0.0) (3.7)
Requirement already satisfied: certifi>=2017.4.17 in /home/{username}/.virtualenvs/r-reticulate/lib/python3.11/site-packages (from requests<3.0,>=2.22.0->synapseclient==4.0.0) (2024.7.4)
Collecting zipp>=0.5 (from importlib-metadata<7.0,>=6.0->opentelemetry-api~=1.21.0->synapseclient==4.0.0)
  Using cached zipp-3.19.2-py3-none-any.whl.metadata (3.6 kB)
Using cached synapseclient-4.0.0-py3-none-any.whl (230 kB)
Using cached Deprecated-1.2.14-py2.py3-none-any.whl (9.6 kB)
Using cached opentelemetry_api-1.21.0-py3-none-any.whl (57 kB)
Using cached opentelemetry_exporter_otlp_proto_http-1.21.0-py3-none-any.whl (16 kB)
Using cached opentelemetry_exporter_otlp_proto_common-1.21.0-py3-none-any.whl (17 kB)
Using cached opentelemetry_proto-1.21.0-py3-none-any.whl (50 kB)
Using cached opentelemetry_sdk-1.21.0-py3-none-any.whl (105 kB)
Using cached opentelemetry_semantic_conventions-0.42b0-py3-none-any.whl (36 kB)
Using cached urllib3-1.26.19-py2.py3-none-any.whl (143 kB)
Using cached backoff-2.2.1-py3-none-any.whl (15 kB)
Using cached googleapis_common_protos-1.63.2-py2.py3-none-any.whl (220 kB)
Using cached importlib_metadata-6.11.0-py3-none-any.whl (23 kB)
Using cached typing_extensions-4.12.2-py3-none-any.whl (37 kB)
Using cached wrapt-1.16.0-cp311-cp311-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_17_x86_64.manylinux2014_x86_64.whl (80 kB)
Using cached protobuf-4.25.4-cp37-abi3-manylinux2014_x86_64.whl (294 kB)
Using cached zipp-3.19.2-py3-none-any.whl (9.0 kB)
Installing collected packages: zipp, wrapt, urllib3, typing-extensions, protobuf, opentelemetry-semantic-conventions, backoff, opentelemetry-proto, importlib-metadata, googleapis-common-protos, deprecated, opentelemetry-exporter-otlp-proto-common, opentelemetry-api, opentelemetry-sdk, opentelemetry-exporter-otlp-proto-http, synapseclient
  Attempting uninstall: urllib3
    Found existing installation: urllib3 2.2.2
    Uninstalling urllib3-2.2.2:
      Successfully uninstalled urllib3-2.2.2
Successfully installed backoff-2.2.1 deprecated-1.2.14 googleapis-common-protos-1.63.2 importlib-metadata-6.11.0 opentelemetry-api-1.21.0 opentelemetry-exporter-otlp-proto-common-1.21.0 opentelemetry-exporter-otlp-proto-http-1.21.0 opentelemetry-proto-1.21.0 opentelemetry-sdk-1.21.0 opentelemetry-semantic-conventions-0.42b0 protobuf-4.25.4 synapseclient-4.0.0 typing-extensions-4.12.2 urllib3-1.26.19 wrapt-1.16.0 zipp-3.19.2
Error: ModuleNotFoundError: No module named 'synapseclient'
Execution halted
ERROR: configuration failed for package ‘synapser’
* removing ‘/home/{username}/R/x86_64-pc-linux-gnu-library/4.3/synapser’
Warning in install.packages :
  installation of package ‘synapser’ had non-zero exit status

The downloaded source packages are in
    ‘/tmp/RtmpETNKZo/downloaded_packages’

Output of sessionInfo()

> sessionInfo()
R version 4.3.2 (2023-10-31)
Platform: x86_64-pc-linux-gnu (64-bit)
Running under: Ubuntu 22.04.4 LTS

Matrix products: default
BLAS:   /usr/lib/x86_64-linux-gnu/openblas-pthread/libblas.so.3 
LAPACK: /usr/lib/x86_64-linux-gnu/openblas-pthread/libopenblasp-r0.3.20.so;  LAPACK version 3.10.0

locale:
 [1] LC_CTYPE=en_US.UTF-8       LC_NUMERIC=C               LC_TIME=en_US.UTF-8        LC_COLLATE=en_US.UTF-8     LC_MONETARY=en_US.UTF-8   
 [6] LC_MESSAGES=en_US.UTF-8    LC_PAPER=en_US.UTF-8       LC_NAME=C                  LC_ADDRESS=C               LC_TELEPHONE=C            
[11] LC_MEASUREMENT=en_US.UTF-8 LC_IDENTIFICATION=C       

time zone: Etc/UTC
tzcode source: system (glibc)

attached base packages:
[1] stats     graphics  grDevices utils     datasets  methods   base     

other attached packages:
[1] reticulate_1.28

loaded via a namespace (and not attached):
 [1] digest_0.6.34     fastmap_1.1.1     xfun_0.45         Matrix_1.6-1.1    lattice_0.21-9    knitr_1.45        htmltools_0.5.7  
 [8] png_0.1-8         rmarkdown_2.25    lifecycle_1.0.4   cli_3.6.2         grid_4.3.2        rsconnect_1.3.1   compiler_4.3.2   
[15] rstudioapi_0.15.0 tools_4.3.2       evaluate_0.23     yaml_2.3.8        Rcpp_1.0.13       rlang_1.1.3       jsonlite_1.8.8
timothytsai commented 1 month ago

Resolved, make sure your .Rprofile doesn't specify a RETICULATE_PYTHON path.

thomasyu888 commented 1 month ago

Thanks @timothytsai for filing this ticket. Sage had an all site week off this week, but we filed an internal ticket to update the documentation in the future. Glad it worked!