search

clj-python / libpython-clj

Python bindings for Clojure
Eclipse Public License 2.0
1.05k stars 68 forks source link

Unable to require libraries from within docker using conda #245

Closed metasoarous closed 1 year ago

metasoarous commented 1 year ago

I set up a repo here to reproduce: https://github.com/metasoarous/libpython-clj-test https://github.com/metasoarous/libpython-clj-conda-docker

The basic code seems to work fine, but when I try to require numpy, I get the error ModuleNotFoundError: No module named 'numpy'.

Any idea what might be going wrong here? I made sure to set the LD_LIBRARY_PATH based on output of python3-config --prefix.

I'm forgetting all the details, but the reason I'm using conda vs just pip is that it was the only way I could get certain python libraries that I need installed.

Thanks in advance!

jjtolton commented 1 year ago

When you run your python command with the same arguments, what do you get for import numpy?

jjtolton commented 1 year ago

Where I'm going with this is we can do a little more work to specify the correct Python environment, whichever is being pointed to does not have numpy installed.

metasoarous commented 1 year ago

Ah! Thanks so much! You're suggestion of testing import numpy from python led to the issue.

Turns out that calling

SHELL ["conda", "run", "-n", "pyclj", "/bin/bash", "-c"]

from the Dockerfile does apply the specified conda environment (pyclj in this case) to all subsequent RUN commands, but not to the final CMD :laughing: Took me a minute to figure out, but simply calling

CMD ["conda", "run", "-n", "pyclj", "clojure", "-M:cider-nrepl"]

did the trick here.

Will update that repo tomorrow as an example of how to use libpython-clj + conda + docker together in case its useful for others to point to.

Thanks again!

jjtolton commented 1 year ago

Great catch! You got that quick, those container issues can be a nightmare.

metasoarous commented 12 months ago

Yes indeed; Worth noting that I got to this point after debugging about a half dozen other issues since the last time I had a working thing :facepalm:

FWIW, I just updated the repo with the fix and some documentation to make it clearer what's going on. You're welcome to merge it into your examples repository or point to it from somewhere if you like.

https://github.com/metasoarous/libpython-clj-conda-docker

Thanks again!

jjtolton commented 12 months ago

Much appreciated. Yes, the example repo is overdue for an overhaul.