RFC: Use `compiler('cuda')`

[conda-forge-admin]

Update recipe to use compiler('cuda') to configure CUDA builds. Adds content needed to conda-build-config.yaml as pulled from conda-forge-pinning for the specific CUDA & CentOS versions used on the different architectures.

---

Hi! This is the friendly automated conda-forge-webservice.

I've rerendered the recipe as instructed in #120.

Here's a checklist to do before merging.

• [x] Bump the build number if needed.

Fixes #120

[conda-forge-webservices[bot]]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[jakirkham]

@conda-forge-admin, please re-render.

[jakirkham]

@conda-forge-admin, please re-render.

[jakirkham]

@conda-forge-admin, please re-render.

[jakirkham]

@conda-forge-admin, please re-render.

[github-actions[bot]]

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/openmpi-feedstock/actions/runs/4542296552.

[jakirkham]

@conda-forge-admin, please re-render.

[jakirkham]

@conda-forge-admin, please re-render.

[jakirkham]

@conda-forge-admin, please re-render.

[leofang]

FYI we're working on this in #128. I can manually backport the changes to the v4.x branch later.

One question for @jakirkham: Why do we need to add a CUDA compiler here, as opposed to using the existing CUDA version matrix conda-build-config.yml?

[jakirkham]

The conda_build_config.yml can run into issues with migrators (especially if they touch the same things like CUDA or compiler versions). Also conda_build_config.yml overrides global pinnings, which can delay updates until they are manually added to conda_build_config.yml. Lastly {{ compiler("cuda") }} does some configuration of build tooling, which can simplify work within recipes (though nothing is perfect). Was trying to just use {{ compiler("cuda") }} for these reasons

Also as the ucx package no longer comes in CPU/GPU flavors, was trying to always use ucx on Linux (and drop some of the special handling around it)

[leofang]

So the reason we used custom cbc.yaml was because we wanted to use the lowest supported CUDA version on CF at the time to build Open MPI, so as to relax the runtime CUDA requirement. But, back then there was no such thing as CUDA minor version compatibility, and now that we have it we can give it a try. That is, switch to use compiler('cuda') and build with CUDA 11.8, so that we can still serve all users with driver 11.0+ (forgive me for not copying/pasting the exact driver number here, it's irrelevant and can be looked up from CUDA compat docs).

[jakirkham]

@conda-forge-admin , please re-render

[github-actions[bot]]

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you but ran into some issues. Please check the output logs of the latest rerendering GitHub actions workflow run for errors. You can also ping conda-forge/core for further assistance or try re-rendering locally.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/openmpi-feedstock/actions/runs/7429925887.

[conda-forge-webservices[bot]]

Hi! This is the friendly automated conda-forge-linting service.

I was trying to look for recipes to lint for you, but it appears we have a merge conflict. Please try to merge or rebase with the base branch to resolve this conflict.

Please ping the 'conda-forge/core' team (using the @ notation in a comment) if you believe this is a bug.

[jakirkham]

@conda-forge-admin, please re-render

[conda-forge-webservices[bot]]

Hi! This is the friendly automated conda-forge-linting service.

I just wanted to let you know that I linted all conda-recipes in your PR (recipe) and found it was in an excellent condition.

[jakirkham]

@conda-forge-admin, please re-render

[github-actions[bot]]

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/openmpi-feedstock/actions/runs/7430075191.

[jakirkham]

@conda-forge-admin , please re-render

[jakirkham]

@conda-forge-admin , please re-render

[github-actions[bot]]

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/openmpi-feedstock/actions/runs/7430124920.

[jakirkham]

@conda-forge-admin , please re-render

[github-actions[bot]]

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/openmpi-feedstock/actions/runs/7430193118.

[jakirkham]

@conda-forge-admin , please re-render

[github-actions[bot]]

Hi! This is the friendly automated conda-forge-webservice.

I tried to rerender for you, but it looks like there was nothing to do.

This message was generated by GitHub actions workflow run https://github.com/conda-forge/openmpi-feedstock/actions/runs/7430228568.

[jakirkham]

@conda-forge-admin , please re-render

[jakirkham]

@conda-forge-admin , please re-render

[jakirkham]

It looks like we are using cross-compilation for linux_aarch64 and emulation for linux_ppc64le. Is that expected?

https://github.com/conda-forge/openmpi-feedstock/blob/c577e78eb375d8ea9be3596a15731ff2b5725f67/conda-forge.yml#L1-L3

https://github.com/conda-forge/openmpi-feedstock/blob/c577e78eb375d8ea9be3596a15731ff2b5725f67/conda-forge.yml#L10-L12

[leofang]

It looks like we are using cross-compilation for linux_aarch64 and emulation for linux_ppc64le. Is that expected?

https://github.com/conda-forge/openmpi-feedstock/blob/c577e78eb375d8ea9be3596a15731ff2b5725f67/conda-forge.yml#L1-L3

It is surprising. It seems the migration bot added the cross compiling line above in commit https://github.com/conda-forge/openmpi-feedstock/commit/27833628db6484b8904d4d53888fdc198e9ad3fa. Not sure why it'd do that?

[leofang]

It looks like we are using cross-compilation for linux_aarch64 and emulation for linux_ppc64le. Is that expected?

It is surprising. It seems the migration bot added the cross compiling line above in commit 2783362. Not sure why it'd do that?

Did a bit more digging, it seems up to the last PR that we merged (#135) we still ran emulation for linux_aarch64. Also, I recalled I didn't quite figure out how to cross-compile Open MPI (#86), so it shouldn't be the default. However, the bot commit was merged 10 months ago, so the only difference that I can think of is someone somehow switching to the CUDA compiler triggers this issue (that should have surfaced earlier). Let me force using QEMU and retry.

[leofang]

@conda-forge-admin, please rerender

[jakirkham]

Thanks Leo! 🙏

Figured some wires were getting crossed in linux_aarch64. Just wasn't sure which way to untangle them. Thanks for sorting that out 🙂

[jakirkham]

@dalcinl what do you think about this PR?

[leofang]

Thanks John & Lisandro! Let's merge.

[jakirkham]

Thank you both! 🙏