Open ganghyun-y opened 1 year ago
Thanks for sharing this and for troubleshooting installation with me.
The first set of warning messages are an R package installation issue (see https://github.com/r-lib/remotes/issues/368). It's best to install R packages in a fresh R session before any are loaded (when possible). I'm not sure if those are causing issues for you but it's a different issue than the python related error from DO.utils.
It seems that the pyDOID python package is still not available to DO.utils. Can you tell me a bit more about your Python setup? Did you have an installation of Python before installing DO.utils?
Can you share with me the results of running reticulate::py_discover_config()
?
I installed the package in a virtual environment called myenv using miniconda. The code below is a list of packages installed in my virtual environment (myenv).
# packages in environment at C:\Users\ykh24\miniconda3\envs\myenv:
#
# Name Version Build Channel
adeft 0.11.2 pypi_0 pypi
aniso8601 9.0.1 pypi_0 pypi
appdirs 1.4.4 pypi_0 pypi
asttokens 2.2.1 pypi_0 pypi
attrs 22.2.0 pypi_0 pypi
backcall 0.2.0 pypi_0 pypi
beautifulsoup4 4.11.2 pypi_0 pypi
bioregistry 0.6.83 pypi_0 pypi
bioversions 0.5.83 pypi_0 pypi
blas 1.0 mkl
boto3 1.26.79 pypi_0 pypi
botocore 1.29.79 pypi_0 pypi
ca-certificates 2023.01.10 haa95532_0
cachier 2.0.0 pypi_0 pypi
certifi 2022.12.7 py39haa95532_0
charset-normalizer 3.0.1 pypi_0 pypi
chembl-downloader 0.4.2 pypi_0 pypi
class-resolver 0.4.0 pypi_0 pypi
click 8.1.3 pypi_0 pypi
click-default-group 1.2.2 pypi_0 pypi
cmake 3.25.2 pypi_0 pypi
colorama 0.4.6 pypi_0 pypi
curies 0.4.2 pypi_0 pypi
dataclasses-json 0.5.7 pypi_0 pypi
decorator 5.1.1 pypi_0 pypi
defusedxml 0.7.1 pypi_0 pypi
dominate 2.7.0 pypi_0 pypi
drugbank-downloader 0.1.1 pypi_0 pypi
enum34 1.1.10 pypi_0 pypi
executing 1.2.0 pypi_0 pypi
fairsharing-client 0.1.0 pypi_0 pypi
flask 2.1.3 pypi_0 pypi
flask-bootstrap 3.3.7.1 pypi_0 pypi
flask-restx 1.0.6 pypi_0 pypi
flask-wtf 1.1.1 pypi_0 pypi
future 0.18.3 pypi_0 pypi
gilda 0.10.3 pypi_0 pypi
gitdb 4.0.10 pypi_0 pypi
gitpython 3.1.31 pypi_0 pypi
humanize 4.6.0 pypi_0 pypi
idna 3.4 pypi_0 pypi
ijson 3.2.0.post0 pypi_0 pypi
importlib-metadata 6.0.0 pypi_0 pypi
indra 1.22.0 pypi_0 pypi
intel-openmp 2021.4.0 haa95532_3556
ipython 8.10.0 pypi_0 pypi
isodate 0.6.1 pypi_0 pypi
itsdangerous 2.1.2 pypi_0 pypi
jedi 0.18.2 pypi_0 pypi
jinja2 3.1.2 pypi_0 pypi
jmespath 1.0.1 pypi_0 pypi
joblib 1.2.0 pypi_0 pypi
jsonschema 4.17.3 pypi_0 pypi
lxml 4.9.2 pypi_0 pypi
markupsafe 2.0.1 pypi_0 pypi
marshmallow 3.19.0 pypi_0 pypi
marshmallow-enum 1.5.1 pypi_0 pypi
matplotlib-inline 0.1.6 pypi_0 pypi
mkl 2021.4.0 haa95532_640
mkl-service 2.4.0 py39h2bbff1b_0
mkl_fft 1.3.1 py39h277e83a_0
mkl_random 1.2.2 py39hf11a4ad_0
more-click 0.1.2 pypi_0 pypi
more-itertools 9.0.0 pypi_0 pypi
mpmath 1.2.1 pypi_0 pypi
mypy-extensions 1.0.0 pypi_0 pypi
ndex2 2.0.1 pypi_0 pypi
networkx 3.0 pypi_0 pypi
nltk 3.8.1 pypi_0 pypi
numpy 1.23.5 py39h3b20f71_0
numpy-base 1.23.5 py39h4da318b_0
objectpath 0.6.1 pypi_0 pypi
obonet 0.3.1 pypi_0 pypi
openssl 1.1.1t h2bbff1b_0
packaging 23.0 pypi_0 pypi
pandas 1.2.5 pypi_0 pypi
parso 0.8.3 pypi_0 pypi
pathtools 0.1.2 pypi_0 pypi
pickleshare 0.7.5 pypi_0 pypi
pip 23.0.1 pypi_0 pypi
portalocker 2.7.0 pypi_0 pypi
prompt-toolkit 3.0.37 pypi_0 pypi
protmapper 0.0.27 pypi_0 pypi
psycopg2-binary 2.9.5 pypi_0 pypi
pure-eval 0.2.2 pypi_0 pypi
pybiopax 0.1.3 pypi_0 pypi
pydantic 1.10.5 pypi_0 pypi
pydoid 0.1.6 pypi_0 pypi
pygments 2.14.0 pypi_0 pypi
pyobo 0.8.12 pypi_0 pypi
pyparsing 3.0.9 pypi_0 pypi
pyrsistent 0.19.3 pypi_0 pypi
pysb 1.14.0 pypi_0 pypi
pysolr 3.9.0 pypi_0 pypi
pystow 0.5.0 pypi_0 pypi
python 3.9.16 h6244533_0
python-dateutil 2.8.2 pypi_0 pypi
pytrie 0.4.0 pypi_0 pypi
pytz 2022.7.1 pypi_0 pypi
pywin32 305 pypi_0 pypi
pyyaml 6.0 pypi_0 pypi
rdflib 6.2.0 pypi_0 pypi
regex 2022.10.31 pypi_0 pypi
requests 2.28.2 pypi_0 pypi
requests-ftp 0.3.1 pypi_0 pypi
requests-toolbelt 0.10.1 pypi_0 pypi
s3transfer 0.6.0 pypi_0 pypi
scikit-learn 1.2.1 pypi_0 pypi
scipy 1.10.1 pypi_0 pypi
setuptools 67.4.0 pypi_0 pypi
six 1.16.0 pyhd3eb1b0_1
smmap 5.0.0 pypi_0 pypi
sortedcontainers 2.4.0 pypi_0 pypi
soupsieve 2.4 pypi_0 pypi
sqlite 3.40.1 h2bbff1b_0
stack-data 0.6.2 pypi_0 pypi
sympy 1.11.1 pypi_0 pypi
tabulate 0.9.0 pypi_0 pypi
threadpoolctl 3.1.0 pypi_0 pypi
tqdm 4.64.1 pypi_0 pypi
traitlets 5.9.0 pypi_0 pypi
typing-extensions 4.5.0 pypi_0 pypi
typing-inspect 0.8.0 pypi_0 pypi
tzdata 2022g h04d1e81_0
umls-downloader 0.1.1 pypi_0 pypi
unidecode 1.3.6 pypi_0 pypi
urllib3 1.26.14 pypi_0 pypi
vc 14.2 h21ff451_1
visitor 0.1.3 pypi_0 pypi
vs2015_runtime 14.27.29016 h5e58377_2
watchdog 2.3.0 pypi_0 pypi
wcwidth 0.2.6 pypi_0 pypi
werkzeug 2.1.2 pypi_0 pypi
wheel 0.38.4 py39haa95532_0
wincertstore 0.2 py39haa95532_2
wtforms 3.0.1 pypi_0 pypi
zenodo-client 0.3.0 pypi_0 pypi
zipp 3.15.0 pypi_0 pypi
I installed Python before, and the image below shows python environments that can be linked to R.
> reticulate::py_discover_config()
python: C:/Users/ykh24/miniconda3/envs/myenv/python.exe
libpython: C:/Users/ykh24/miniconda3/envs/myenv/python39.dll
pythonhome: C:/Users/ykh24/miniconda3/envs/myenv
version: 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Users/ykh24/miniconda3/envs/myenv/Lib/site-packages/numpy
numpy_version: 1.23.5
Hmmm... that is strange. It seems like you should be good to go. I have noticed in the past that reticulate will sometimes activate the wrong environment and that may be what it's doing here. The only way I've found to get reticulate to use a consistent environment is to ensure the environment variable RETICULATE_PYTHON is set. This setting seems to override everything else.
Can you provide the output of reticulate::py_config()
? Can you also share the result of reticulate::py_list_packages()
? I know these seem redundant with what you've already sent but this will tell you which python environment is actually being activated (as opposed to those that are available) and whether it is "forced" by the RETICULATE_PYTHON environment variable and then list the of packages included in the loaded environment.
If you restart R to get a fresh session, then set the RETICULATE_PYTHON variable with Sys.setenv(RETICULATE_PYTHON = "C:/Users/ykh24/miniconda3/envs/myenv/python.exe")
and then run reticulate::py_config()
do you get a different result?
Looking at the reticulate issue tracker there are quite a few issues surrounding installation and some of these have been resolved in later releases than the 1.23 release used by DO.utils, so I'm going to bump the version of reticulate used by DO.utils to the latest release (v1.28) in the hope that it will improve things.
In general, I've been keeping an eye on tensorflow for how to ensure proper install. I noticed that they are still using an approach for installation that relies on a custom install function (install_tensorflow()
). I was previously under the impression that they had moved to the automated python installation provided using the package DESCRIPTION file but that does not appear to be the case. I may need to add a custom install function to handle python dependencies as well.
Sorry you're having so much trouble with this.
This appears to be a problem a number of R packages that depend on python packages are having (https://github.com/rstudio/reticulate/issues/997).
I need to check whether updating the reticulate version has fixed the problem but if it didn't I discovered a pretty simple workaround. Anyone who installs miniconda when prompted should have an r-reticulate
conda environment automatically created (running reticulate::py_config()
can confirm this; the python path will include 'envs/r-reticulate' in it). In this case, all that's needed is to install the needed python packages manually into the environment by running reticulate::py_install("pyDOID", pip = TRUE)
.
I tried to reticulate::py_install("pyDOID", pip = TRUE)
, but it still can't find the pyDOID module.
Does it mean that DO.utils needs to be reinstalled in a virtual environment called r-retriculate?
DO.utils doesn't need to be in any specific environment but the python package(s) it sometimes uses do. I tested this recently on a Mac and it automatically created the r-reticulate virtual environment but didn't complete the install of pyDOID. It did work for me to simply tell it to install. In order to install more packages, you do have to have a virtual environment created and activated but it looks like you already do. Yours is just named myenv.
Did the install of pyDOID seem to work when you ran reticulate::py_install("pyDOID", pip = TRUE)
? Or did it immediately produce error messages?
Since you had issues with the initial install and had all those 'problem copying...' errors it's possible that you have some corrupted files that are causing issues. It might be best to try re-installing from scratch. I'll try installing DO.utils on a PC later in the week and let you know what I get.
yes, I did.
I get a message that it's already installed, but it still doesn't work. Next time, I will completely delete the package and proceed with the re-installation.
Thank you for your help. It's hard work, so I hope it's solved well.
> reticulate::py_config()
python: C:/Users/ykh24/miniconda3/envs/r-reticulate/python.exe
libpython: C:/Users/ykh24/miniconda3/envs/r-reticulate/python39.dll
pythonhome: C:/Users/ykh24/miniconda3/envs/r-reticulate
version: 3.9.16 (main, Jan 11 2023, 16:16:36) [MSC v.1916 64 bit (AMD64)]
Architecture: 64bit
numpy: C:/Users/ykh24/miniconda3/envs/r-reticulate/Lib/site-packages/numpy
numpy_version: 1.23.5
NOTE: Python version was forced by use_python function
> reticulate::py_install("pyDOID", pip = TRUE)
C:\Users\ykh24\Desktop\project\test_conda>CALL "C:\Users\ykh24\miniconda3\condabin\activate.bat" "C:\Users\ykh24\miniconda3\envs\r-reticulate"
C:\Users\ykh24\Desktop\project\test_conda>conda.bat activate "C:\Users\ykh24\miniconda3\envs\r-reticulate"
(r-reticulate) C:\Users\ykh24\Desktop\project\test_conda>"C:/Users/ykh24/miniconda3/envs/r-reticulate/python.exe" -m pip install --upgrade --no-user "pyDOID"
Requirement already satisfied: pyDOID in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (0.1.6)
Requirement already satisfied: GitPython in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from pyDOID) (3.1.31)
Requirement already satisfied: tqdm in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from pyDOID) (4.64.1)
Requirement already satisfied: rdflib in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from pyDOID) (6.2.0)
Requirement already satisfied: pandas in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from pyDOID) (1.2.5)
Requirement already satisfied: gitdb<5,>=4.0.1 in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from GitPython->pyDOID) (4.0.10)
Requirement already satisfied: numpy>=1.16.5 in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from pandas->pyDOID) (1.23.5)
Requirement already satisfied: python-dateutil>=2.7.3 in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from pandas->pyDOID) (2.8.2)
Requirement already satisfied: pytz>=2017.3 in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from pandas->pyDOID) (2022.7.1)
Requirement already satisfied: pyparsing in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from rdflib->pyDOID) (3.0.9)
Requirement already satisfied: isodate in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from rdflib->pyDOID) (0.6.1)
Requirement already satisfied: setuptools in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from rdflib->pyDOID) (67.4.0)
Requirement already satisfied: colorama in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from tqdm->pyDOID) (0.4.6)
Requirement already satisfied: smmap<6,>=3.0.1 in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from gitdb<5,>=4.0.1->GitPython->pyDOID) (5.0.0)
Requirement already satisfied: six>=1.5 in c:\users\ykh24\miniconda3\envs\r-reticulate\lib\site-packages (from python-dateutil>=2.7.3->pandas->pyDOID) (1.16.0)
Hi, This warning message (above) appears after installing DO.utils. There was nothing wrong with loading the DO.utils library. It seemed to be going well, but there seemed to be a problem with installing the "adeft" packages, so I manually proceeded with setup.py and solved it.
The RStudio also outputs TRUE.
It seemed like the package problem was being solved, but... It doesn't work :(