search

conda-forge / conda-forge.github.io

The conda-forge website.
https://conda-forge.org
BSD 3-Clause "New" or "Revised" License
131 stars 274 forks source link

Where does conda-forge specify the minimum supported CPU instruction set, e.g. `-march=nocona` #2251

Open corneliusroemer opened 3 months ago

corneliusroemer commented 3 months ago

Your question:

A bioconda recipe I maintain caused some issues by using AVX instructions. The source code build script hard coded -mavx ignoring the default -march flag of gcc and clang, which are at the time of writing -march=core2 -mssse3 or -march=nocona.

The fact that there are default arch flags took some time to figure out, because what the flags are does not seem to be documented explicitly (at least a few folks at bioconda weren't aware, see Gitter thread). I was wondering if we could improve the situation around this.

If I'm not mistaken, there is no single central place (like for example conda-forge-pinnings) setting those flags across compilers in a consistent manner. Instead, each compiler compiler sets its own flags in its feedstock code. Examples:

For x86-64 and ppc but not for arm/aarch, there's been work on allowing build-variants for differing instruction sets: https://github.com/conda-forge/microarch-level-feedstock/blob/1b446dbab3999c5e27f1e0b000231f18fa1b6df1/recipe/install_scripts.sh#L7

A few other mentions of -march I found through search:

A lot of feedstocks override -march in inconsistent ways. Should this be discouraged - unless the package supports graceful degradation when run on lesser hardware? I've seen such recommendations in various issues, maybe this should be made explicit in the docs to make it more discoverable?

The minimum supported architectures have not been updated in a while, should they maybe be?

Here's a collection of somewhat related issues I could find:

CentOS will raise minimum microarch level to x86-64-v2 in version 9 - but end of support of 8 is years away, probably after 2030.

jaimergp commented 2 months ago

To the best of my knowledge, this is a perfect summary of the current situation. This would be an excellent topic to cover in the conda-forge calls.

corneliusroemer commented 2 months ago

@jaimergp - thanks for confirming my investigation didn't go in the wrong direction. Calls are nice, but I don't attend them and so won't many people who might like to know answers to such questions. Written (and search engine indexed) summaries would be very useful in addition!

jaimergp commented 1 month ago

Yep, the idea is to get some feedback there (because it spans multiple pieces of our infra) and then consolidate the outcome in this issue and, eventually, document it / adjust code as needed.

I added it to our tomorrow's agenda and will present it on your behalf, if that's ok.

jaimergp commented 1 month ago

Ok @corneliusroemer, better late than never, but I brought this topic up in today's core. Basically everything here seems to be correct, so it would be awesome to have it as part of the documentation. Maybe after this section?

beckermr commented 1 month ago

This choice of flag goes waaay back. We'd have to ask some old timers why they picked this versus something else.

hmaarrfk commented 1 month ago

I thought we had the ability to improve performance with archspec. it creates a whole build matrix, but maybe worthwhile to look into for your usecase?

https://github.com/conda-forge/conda-forge.github.io/issues/1261

beckermr commented 1 month ago

We do @hmaarrfk, but that is not original to conda. The choice of nocona as the min was my by anaconda inc ages ago.