search

conda-forge / mpich-feedstock

A conda-smithy repository for mpich.
BSD 3-Clause "New" or "Revised" License
2 stars 26 forks source link

ENH add shim for external MPI #44

Closed beckermr closed 5 years ago

beckermr commented 5 years ago

Checklist

This PR adds a shim for external MPI distributions per https://github.com/conda-forge/staged-recipes/pull/9536 and https://github.com/conda-forge/conda-forge.github.io/issues/860.

conda-forge-linter commented 5 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

beckermr commented 5 years ago

@conda-forge-admin, please rerender

beckermr commented 5 years ago

@conda-forge-admin, please rerender

conda-forge-linter commented 5 years ago

Hi! This is the friendly automated conda-forge-webservice.

I tried to re-render for you, but it looks like there was nothing to do.

beckermr commented 5 years ago

@conda-forge-admin, please rerender

beckermr commented 5 years ago

OK @minrk @isuruf. Can you all take a look at this version of the shims?

Given that mpich has an initiative around ABI compatibility, we might just merge this and pass on supporting openmpi for the time being.

beckermr commented 5 years ago

Bump here @conda-forge/mpich

dalcinl commented 5 years ago

@beckermr I have no idea about how this external MPI support is actually implemented, so I'm not qualified to review your changes. Do you want me to merge this one right away?

beckermr commented 5 years ago

Yes please @dalcinl! This one is ready to merge!

minrk commented 5 years ago

Let's merge this one, which will create the 3.3.1 build. If you can report back on how it is working for you, we can make sure that a few past versions get uploaded as well, if that seems useful.

One thing that may prove useful: a post-link script that verifies it can be found and is an appropriate version (e.g. checking for libmpi.so.12), rather than just an empty package.

leofang commented 4 years ago

@beckermr Two questions:

  1. How to use the "external" feature if there's already an MPI library outside the conda env? Is it documented somewhere?
  2. Could you propagate this change to openmpi-feedstock?
beckermr commented 4 years ago

Hmmmm. I bet I didn't write docs. Install the proper build with the feature and it should just work.

We cannot so openmpi because the ABI stability there is pretty bad.

beckermr commented 4 years ago

Something like

conda install mpich==*=external_*

should be close to what you need.

leofang commented 4 years ago

Thanks, @beckermr!

Hmmmm. I bet I didn't write docs. Install the proper build with the feature and it should just work.

I found there is a CF doc section on MPI: https://conda-forge.org/docs/maintainer/knowledge_base.html#message-passing-interface-mpi What's the right repo to submit a PR?

We cannot so openmpi because the ABI stability there is pretty bad.

Oh...I know there's not ABI-compatibility guarantee in Open MPI, but I thought your earlier comment https://github.com/conda-forge/conda-forge.github.io/issues/860#issuecomment-530976401 suggested it's still doable. I probably parsed it wrong...

Still, couldn't we at least make it available for any downstream conda packages? I imagine something like this should work:

conda install openmpi=*=*external* mpi4py=*=*openmpi* h5py=*=*openmpi*

Since Open MPI and MPICH are mutually exclusive through the mpi meta package, it should be enough to avoid the ABI compatible issue, don't you think?

beckermr commented 4 years ago

Sorry I was not clear. openmpi is not abi compatible with itself. What we are doing here is compiling against one version of the libs (our mpich) and then linking code against another. openmpi may not work doing that. mpich has a much better chance.

leofang commented 4 years ago

openmpi is not abi compatible with itself

I see. I was not aware of that. Thanks, Matt!