Open snickell opened 1 month ago
Output from PYCALL_DEBUG_FIND_LIBPYTHON=1:
DEBUG(find_libpython) find_libpython("/Users/seth/src/code-dot-org/.venv/bin/python")
DEBUG(find_libpython) investigate_python_config("/Users/seth/src/code-dot-org/.venv/bin/python")
DEBUG(find_libpython) Candidate: /opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/Python
DEBUG(find_libpython) Trying to dlopen: /opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/Python
DEBUG(find_libpython) dlopen("/opt/homebrew/Cellar/python@3.12/3.12.4/Frameworks/Python.framework/Versions/3.12/Python") = #<Fiddle::Handle:0x0000000120b93798>
virtualenv is linking to /opt/homebrew python:
ls -l /Users/seth/src/code-dot-org/.venv/bin | grep python
# => python -> /opt/homebrew/opt/python@3.12/bin/python3.12
Further investigation:
sys.executable
is returning the path to ruby: e.g. '/Users/seth/.rbenv/versions/3.0.5/bin/ruby', probably because we are using libpython not a python exe.If a file named "pyvenv.cfg" exists one directory above sys.executable, sys.prefix and sys.exec_prefix are set to that directory and it is also checked for site-packages (sys.base_prefix and sys.base_exec_prefix will always be the "real" prefixes of the Python installation).
I have noticed that "editable" package installations (site-packages/*.pth files) are also not registered into the path from pycall.rb. This is consistent with site.py
being invoked and not finding pyenv.cfg (present in my .venv directory). As a result, local site-packages are not properly registered.
I wonder: is there a way to set sys.executable to the local python bin? This would make site.py function as expected.
Here's a "semi proper" workaround that anyone else facing this can do:
path_to_your_venv = '/Users/seth/src/code-dot-org/.venv/lib/python3.12/site-packages' # your venv path here
require 'pycall'
site = PyCall.import_module('site')
site.addsitedir(path_to_your_venv)
This will properly setup your venv's site-packages, including registering .pth files (editable installs) etc.
I'm trying to use a virtualenv with pycall. Python is loaded correctly, but sys.path does not include the site-packages from the virtualenv. Instead site-packages from the underlying python interpreter (in /opt/homebrew) is used.
For example, lets say I have a new version of
urllib3
in my virtualenv:If I manually set PYTHONPATH, it will load from the virtualenv:
The problem with setting PYTHONPATH manually, is now I cannot invoke 3rd party python-using tools like
aws
cli from my process, because the manual PYTHONPATH will point them at my virtualenv.