search

rstudio / reticulate

R Interface to Python
https://rstudio.github.io/reticulate
Apache License 2.0
1.68k stars 327 forks source link

Possible to change libpython path in `py_discover_config()`? #307

Open kch382001 opened 6 years ago

kch382001 commented 6 years ago

I built python under my personal account, then packaged it and distributed the binary to different servers. however I keep getting error running py_config() as below: Error in initialize_python(required_module, use_environment) : Python shared library '/home/kec5478/Python-3.6.4_build/lib/libpython3.6.so' not found, Python bindings not loaded

it turned out that "reticulate" deduced the shared library path based on the built account from the python binary. See py_discover_config() below:

py_discover_config() python: /mnt/netapp_hfalgo/apps/hfalgo_ext/ec88300/python/Python-3.6.4/bin/python libpython: /home/kec5478/Python-3.6.4_build/lib/libpython3.6.so[NOT FOUND] pythonhome: /home/kec5478/Python-3.6.4_build:/home/kec5478/Python-3.6.4_build version: 3.6.4 (default, Jan 17 2018, 17:24:22) [GCC 7.2.0] numpy: /mnt/netapp_hfalgo/apps/hfalgo_ext/ec88300/python/Python-3.6.4/lib/python3.6/site-packages/numpy numpy_version: 1.14.0

My question is: how to handle this situation? If there is an API to change libpython path and pythonhome, that would solve the problem.

jjallaire commented 6 years ago

I don't think this will work as Python itself is reporting the location within your home directory (there is no way for us to know the location of the Python shared library w/o asking Python, as it varies considerably depending on how Python was built).

kch382001 commented 6 years ago

The python built within my home directory then distributed to a different account on server box actually worked fine except that "reticulate" does not work because the paths deduced from python binary point to the build directory structure. I am wondering if it is possible to configure the build so that python binary remembers the relative path of the shared lib instead of absolute path. Then reticulate can resolve the relative path on any machine when it deduces from python binary code. First of all, I do not know if that is possible or not? Secondly it would be nice to have an API to allow user to manually set the "libpython" and "pythonhome", is that an option?

jjallaire commented 6 years ago

Could you use the --prefix option to specify the alternate path when you build? (see https://www.iram.fr/IRAMFR/GILDAS/doc/html/gildas-python-html/node36.html)

tyronehunt commented 4 years ago

The only thing that fixed this issue for me was creating a new virtual environment. RStudio seems to not like you using the same virtual environment for multiple projects.

Summary of how to create a new virtual environment: python_path = '/usr/local/bin/python3' reticulate::virtualenv_create(envname = 'VIRTUALENV_NAME', python = python_path)

List what you want installed PYTHON_DEPENDENCIES = c('pandas', 'numpy') reticulate::virtualenv_install('VIRTUALENV_NAME', packages = PYTHON_DEPENDENCIES)