search

conda-forge / openmpi-feedstock

A conda-smithy repository for openmpi.
BSD 3-Clause "New" or "Revised" License
9 stars 25 forks source link

openmpi v4.1.0 #71

Closed regro-cf-autotick-bot closed 3 years ago

regro-cf-autotick-bot commented 3 years ago

It is very likely that the current package version for this feedstock is out of date. Notes for merging this PR:

  1. Feel free to push to the bot's branch to update this PR if needed.
  2. The bot will almost always only open one PR per version. Checklist before merging this PR:
    • [ ] Dependencies have been updated if changed: see upstream
    • [ ] Tests have passed
    • [ ] Updated license if changed and license_file is packaged

Note that the bot will stop issuing PRs if more than 3 Version bump PRs generated by the bot are open. If you don't want to package a particular version please close the PR.

NEW: If you want these PRs to be merged automatically, make an issue with code>@conda-forge-admin,</codeplease add bot automerge in the title and merge the resulting PR. This command will add our new bot automerge feature to your feedstock!

If this PR was opened in error or needs to be updated please add the bot-rerun label to this PR. The bot will close this PR and schedule another one. If you do not have permissions to add this label, you can use the phrase code>@<space/conda-forge-admin, please rerun bot in a PR comment to have the conda-forge-admin add it for you.

This PR was created by the regro-cf-autotick-bot. The regro-cf-autotick-bot is a service to automatically track the dependency graph, migrate packages, and propose package version updates for conda-forge. If you would like a local version of this bot, you might consider using rever. Rever is a tool for automating software releases and forms the backbone of the bot's conda-forge PRing capability. Rever is both conda (conda install -c conda-forge rever) and pip (pip install re-ver) installable. Finally, feel free to drop us a line if there are any issues! This PR was generated by https://github.com/regro/autotick-bot/actions/runs/431342026, please use this URL for debugging

Here is a list of all the pending dependencies (and their versions) for this repo. Please double check all dependencies before merging.

Name Upstream Version Current Version
openmpi 4.1.0 Anaconda-Server Badge

Dependency Analysis

We couldn't run dependency analysis due to an internal error in the bot. :( Help is very welcome!

conda-forge-linter commented 3 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

dalcinl commented 3 years ago

@isuruf @leofang I'm not sure what's the proper way to address issues related to use of AVX instructions. I would rather disable them in conda-forge builds, but it is not obvious how to do so via ./configure. Should we ask upstream?

dalcinl commented 3 years ago

@leofang Should we now remove what you added in #58 ?

dalcinl commented 3 years ago

@isuruf In 8bfa1c70, you added ./autogen.pl --force. Is that really needed in all target platforms?

leofang commented 3 years ago

Sorry for my late reply, @dalcinl. I took a quick look and found Homebrew people hit the same issue: https://github.com/open-mpi/ompi/issues/8306. I will raise my hand there.

leofang commented 3 years ago

Thanks a lot for helping us resolve the issue @isuruf! So it looks like ejecting -march and -mtune does stop building the avx512 module on both Linux and macOS. A flag --enable-mca-no-build=op-avx was suggested in https://github.com/open-mpi/ompi/issues/8306, and I'll give it a shot and see how it goes. It it's not working I'll revert to your approach, is that ok?

leofang commented 3 years ago

A flag --enable-mca-no-build=op-avx was suggested in open-mpi/ompi#8306, and I'll give it a shot and see how it goes.

It looks like this flag will stop building the op:avx component altogether whereas @isuruf your approach will still build it, which is in fact preferable. Could you share with me what's going on when ejecting -march and -mtune? Will we ship out a package with avx512 enabled by default? It might be a bit dangerous...

In the meanwhile, I will check out @dalcinl's earlier comment (sorry!)

Should we now remove what you added in #58 ?

isuruf commented 3 years ago

Will we ship out a package with avx512 enabled by default? It might be a bit dangerous

Not really. open-mpi only enables avx512 if the CPU at runtime supports it. So that's fine.

leofang commented 3 years ago

Thanks @isuruf!

@dalcinl Looks like #58 can be safely reverted. All CI passed. PTAL.

dalcinl commented 3 years ago

Sorry, folks, but I'm confused. If we do not ask the compiler for -march, what's the guarantee the binaries will work in the oldest processors conda-forge is supposed to support? Isn't much safer to just avoid messing with -march and instead disable the avx component?

leofang commented 3 years ago

Hi @dalcinl According to OMPI folks, the AVX paths would not be invoked at runtime if it detects the CPU does not support it, see https://github.com/open-mpi/ompi/issues/8306#issuecomment-751744153 and https://github.com/open-mpi/ompi/issues/8306#issuecomment-751807012. Sounds like black magic to me 🙂 but it should be safe then.

dalcinl commented 3 years ago

@leofang Please read my question again. In understand how Open MPI is doing things, that's not my concern. What I'm worried about is that after this PR is merged, all the sources will be compiled without -march flags. These -march flags come from conda-forge and for good reasons (get binaries built using a consistent instruction set), and we are just throwing them away.

isuruf commented 3 years ago

gcc and clang uses the default triplet the compiler was built with if the -march is not passed. Since the compilers we are using were built with x86_64-conda-linux-gnu and x86_64-apple-darwin for linux and osx respectively, the compiler will not use a newer instruction set unless specified.

leofang commented 3 years ago

@conda-forge-admin, please rerender

leofang commented 3 years ago

Incompatible C and Fortran compiler versions...?

leofang commented 3 years ago

@conda-forge-admin, please rerender

github-actions[bot] commented 3 years ago

Hi! This is the friendly automated conda-forge-webservice. I tried to rerender for you, but it looks like there was nothing to do.

leofang commented 3 years ago

Incompatible C and Fortran compiler versions...?

Perhaps https://github.com/conda-forge/conda-forge-pinning-feedstock/pull/1090 could fix this?

leofang commented 3 years ago

Incompatible C and Fortran compiler versions...?

Perhaps conda-forge/conda-forge-pinning-feedstock#1090 could fix this?

I fixed it by locking the Fortran compiler versions in conda_build_config.yaml. This is necessary given the way the present CUDA 11.0 migration is done. We can remove it once the migration finishes.

Earlier today there was an error when running mpiexec --allow-run-as-root ..., but it seems to be a transient thing. Maybe related to the conda-build failure reported earlier today (here and here)?

conda-forge-linter commented 3 years ago

Hi! This is the friendly automated conda-forge-linting service.

I wanted to let you know that I linted all conda-recipes in your PR (recipe) and found some lint.

Here's what I've got...

For recipe:

conda-forge-linter commented 3 years ago

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

leofang commented 3 years ago

@isuruf Thanks. I think it's a right call to do ignore_run_exports_from, as the build here really has nothing to do with CUDA other than having a module built. Am I getting it right that the trick to fix Fortran dependency is to manually remove the CUDA 11 migration?

leofang commented 3 years ago

@conda-forge-admin, please rerender

github-actions[bot] commented 3 years ago

Hi! This is the friendly conda-forge automerge bot!

I considered the following status checks when analyzing this PR:

Thus the PR was passing and merged! Have a great day!

leofang commented 3 years ago

Thanks again @ggouaillardet @bosilca @jsquyres @isuruf @dalcinl for the community effort 🎉

jsquyres commented 3 years ago

Note that we're likely to have at least one more significant AVX-related change on the Open MPI v4.1.x branch in the Very Near Future: see https://github.com/open-mpi/ompi/pull/8373. The title is a little misleading (i.e., you want this even if you don't care about the Intel compilers) -- the 2nd commit which disables AVX2 and AVX512 by default is significant. See https://github.com/open-mpi/ompi/issues/8334 for the (unexpected) backstory.