Unable to use libraries from a venv

[atisharma]

I would like to use a venv to manage python dependencies with libpython-clj. I have been unable to find out how to do this.

I tried the following approaches.

(A) I tried starting a repl from within a venv (using bin/activate). libpython-clj fails initialize with INFO: Library python3.8 found at [:system "python3.8"] Python path configuration: PYTHONHOME = '/tmp/python-interop' PYTHONPATH = (not set) program name = '/tmp/python-interop/bin/python3' isolated = 0 environment = 1 user site = 1 import site = 1 sys._base_executable = '/tmp/python-interop/bin/python3' sys.base_prefix = '/tmp/python-interop' sys.base_exec_prefix = '/tmp/python-interop' sys.executable = '/tmp/python-interop/bin/python3' sys.prefix = '/tmp/python-interop' sys.exec_prefix = '/tmp/python-interop' sys.path = [ '/tmp/python-interop/lib/python38.zip', '/tmp/python-interop/lib/python3.8', '/tmp/python-interop/lib/python3.8/lib-dynload', ] Fatal Python error: init_fs_encoding: failed to get the Python codec of the filesystem encoding Python runtime state: core initialized ModuleNotFoundError: No module named 'encodings'

(B) I tried starting without the venv and setting $PYTHONPATH from leiningen's env-vars. This starts python fine, and I can use (py/initialize!). To set the path to look for venv libraries, I tried (as per require-python docstring):

(require-python 'sys) (py/call-attr (py/get-attr sys "path") "append" ("/tmp/python-interop"))

which imports sys OK but (py/get-attr sys "path") gives Unable to resolve symbol: sys in this context.

(require-python 'os) (os/chdir "/tmp/python-interop")

executes fine, but does not let me use libraries in the venv.

Leiningen 2.9.3 Linux 5.4.45_1 (Void) Python 3.8.3 Clojure 1.10.1 OpenJDK 64-Bit Server VM 11.0.5-internal+11-adhoc..jdk11u-jdk-11.0.510

[jjtolton]

For a regular virtual environment, all you need to do is provide the full path to the python executable.

(py/initialize! :python-executable "/path/to/env/bin/python")

You do NOT need bin/activate -- it will probably cause problems.

Where did you see that docstring? That is almost correct but not entirely.

(require-python 'sys)

If you are coming from Python, you would be tempted to think that this would make the value of sys to the Python's sys module. However, if you are coming from Clojure, you know that (require 'clojure.core.async) does not bind any value to the symbol clojure.core.async -- it will throw an exception if you attempt to evaluate it. We always defer to Clojure's behavior in situations where there is a conflict as a default, however, we do offer ways to get the Pythonic behavior: (require-python '[sys :bind-ns true]) will bind Python's sys module to the symbol sys.

This code would then work as expected:

(py/call-attr (py/get-attr sys "path") "append" "/tmp/python-interop")

(note that ("/tmp/python-interop")) won't work because you have a list literal in there.

However -- since you are using require-python, you can now use the idiomatic Clojure syntax:

(require-python 'sys.path)
(sys.path/append "/tmp/python-interop")

instead!

If you are still having issues feel free to pop into our chat and we'll do our best to help you out :)

https://clojurians.zulipchat.com/#narrow/stream/215609-libpython-clj-dev

[atisharma]

all you need to do is provide the full path to the python executable.

This gives the same result as (A) above.

Where did you see that docstring?

(doc require-python)

you can now use the idiomatic Clojure syntax:

interop.core=> (require-python 'sys.path)
:ok

interop.core=> (sys.path/append "/venv/backtesting")
Syntax error compiling at (/tmp/form-init13488956686171970139.clj:1:1).
No such var: sys.path/append

(require-python '[sys :bind-ns true]) will bind Python's sys module to the symbol sys.

This did work for me and allowed me to import, thank you.

[cnuernber]

This appears to have resolved. Closing (reopen if necessary).

[jjtolton]

Where did you see that docstring?

(doc require-python)

I need to update this -- thank you for pointing that out @atisharma