search

conda-forge / netcdf4-feedstock

A conda-smithy repository for netcdf4.
BSD 3-Clause "New" or "Revised" License
3 stars 25 forks source link

Add mpi4py to build #43

Closed jhamman closed 4 years ago

jhamman commented 6 years ago

There is a switch in netcdf4's setup.py script that disables the parallel IO feature when mpi4py is not available. Would it work to add mpi4py to the build step in this feedstock? I'm personally not sure of the full implications of this so this is really just a question about feasibility.

ocefpaf commented 6 years ago

I believe that the right way to do this would be with build variants, so we can have a parallel and a non-parallel version of netcdf4. I also believe we would need to do the same for netcdf-c.

I can't look into this right now but this is definitely something we should pursue.

jhamman commented 6 years ago

Thanks @ocefpaf.

bekozi commented 6 years ago

I'm :+1: on this. It requires a parallel hdf5 build - looks like that discussion has already been started: https://github.com/conda-forge/hdf5-feedstock/issues/51.

annefou commented 5 years ago

We now have a parallel hdf5 build. How can we proceed to get a parallel netcdf4 build?

ocefpaf commented 5 years ago

We now have a parallel hdf5 build. How can we proceed to get a parallel netcdf4 build?

We need a parallel build of libnetcdf. If you want to give it a try please check https://conda-forge.org/docs/maintainer/knowledge_base.html#message-passing-interface-mpi first.

annefou commented 5 years ago

Ok. I have never done anything more than the very basic with conda but I am happy to learn. I'll look at the documentation you pointed and I guess I can also check what has been done for hdf5.

ocefpaf commented 5 years ago

I guess I can also check what has been done for hdf5.

It should be very similar to hdf5. Feel free to ping me for help along the way.

bgruening commented 5 years ago

xref: https://github.com/conda-forge/hdf5-feedstock/blob/master/recipe/meta.yaml#L38

annefou commented 5 years ago

@ocefpaf sorry to bother you but I don't fully understand how to specify hdf5 in libnetcdf-feedstock. How can I specify we need to use the parallel version of hdf5 when using mpicc and the serial version otherwise?

ocefpaf commented 5 years ago

@annefou it turns out it is not a simple PR. Let me push what I have and you can help me there. Meanwhile please take a look at the docs: https://conda-forge.org/docs/maintainer/knowledge_base.html?highlight=parallel

annefou commented 5 years ago

Thanks for the link to the documentation. So if I understand properly, you add parallel netCDF for openmpi. I start to understand (slowly...). Thanks.

xylar commented 4 years ago

Unless anyone is opposed, I'd like to try to build the MPI matrix for this feedstock.

ocefpaf commented 4 years ago

Yes please! This has been on my to-do list since we got libnetcdf working with mpi.

annefou commented 4 years ago

That would be awesome!