search

NERSC / shifter

Shifter - Linux Containers for HPC
Other
348 stars 65 forks source link

Feature suggestiong: change RPATH to RUNPATH for Anaconda #287

Closed JBlaschke closed 3 years ago

JBlaschke commented 3 years ago

Hi all y'all

this is something that I noticed a few months ago (@dmjacobsen and @scanon might remember a conversation or two slack), and I have iterated since:

I'm trying to find a solution to the following (related) problems:

  1. mpi4py installed in anaconda will not pick up the system's libmpi.so -- even if it depends on the same version
  2. If system libraries (eg. libreadline.so) are a different version than those installed by anaconda, we see warnings of version mismatches (this can fill up stderr).

The current documented solution is to build libraries "by hand" (eg https://docs.nersc.gov/development/shifter/how-to-use/#using-mpi-in-shifter). But this can be a bit tedious: if conda packages depend on libmpi.so, then conda will keep trying to install it. I'm not saying that this doesn't work -- it's just a bit annoying.

So I want to find a way to get conda to not "override" libraries in LD_LIBRARY_PATH. Anaconda packages are built using RPATHs which has higher precedence than the LD_LIBRARY_PATH. I wrote a wee tool that will patch all RPATHs to RUNPATHs here:https://github.com/JBlaschke/conda.local/blob/master/util/patch-rpath.py

I add the following to the Dockerfile:

python /img/conda.local/util/patch-rpath.py /path/to/miniconda3/lib

So far, I've not found any application where this doesn't work. So is there a way to add this step to the image conversion step in shifter? Let me know what you need from me :)

scanon commented 3 years ago

We prefer not to do any content manipulation without the user choosing it. Maybe this is best handled by adding some docs on the NERSC site.

JBlaschke commented 3 years ago

Thanks @scanon -- that makes sense. I'll reach out to you over slack about how best to distribute the patch-rpath.py