dnicolodi
commented
1 year ago I cannot reproduce. Can you please provide a complete build log?
Closed WillAyd closed 1 year ago
dnicolodi
commented
1 year ago I cannot reproduce. Can you please provide a complete build log?
WillAyd
commented
1 year ago Are you asking for the complete compile_commands.json file or something else?
dnicolodi
commented
1 year ago I would like to see the output of the pip install -ve ... command. Also, did you previously use the same build directory with a different meson project configuration?
WillAyd
commented
1 year ago No reuse of the build directory. Here's the output
dnicolodi
commented
1 year ago Nothing strange here. And nothing unusual in the pandas meson.build, at least at a rapid check. Still I cannot reproduce the issue. Let me try to build pandas.
WillAyd
commented
1 year ago Ah I think I figured it out. Mamba/conda mess around with CFLAGS, so meson must be picking that up
dnicolodi
commented
1 year ago Meson obeys $CFLAGS if it is set, yes.
dnicolodi
commented
1 year ago @WillAyd I see a few small problems with the pandas' meson.build. Should I open a PR?
WillAyd
commented
1 year ago For sure - would love any contributions to improve that
WillAyd
commented
1 year ago OK in this particular case it looks like I have to set both CFLAGS and CPPFLAGS
export CFLAGS="$CFLAGS -O0"
export CPPFLAGS="$CPPFLAGS -O0"The reasoning there is clear, but I'm wondering if meson-python should auto-detect if you are using conda/mamba and set these flags for you with debug builds?
dnicolodi
commented
1 year ago I'm wondering if meson-python should auto-detect if you are using conda/mamba and set these flags for you with debug builds?
This fees way to magical for something a build tool should do. I don't even know what a reliable way to detect conda or mamba would be. Let alone the fact that it is impossible to know whether the environment variables are set the way they are because the user means it, or because that's the environment default.
I don't think conda or mamba setting a default value for $CFLAGS is a great idea, anyway.
WillAyd
commented
1 year ago Yea I don't disagree that its confusing for mamba to be doing that. Seems unlikely to change though.
What's interesting is that prior to meson I never had this amount of trouble to get non-optimized extensions. I'm guessing setuptools just ignored those flags, or maybe detected conda like this
eli-schwartz
commented
1 year ago I do recall that setuptools had an unbelievable hodgepodge of flags originally used when compiling CPython itself, which tended to confuse things a lot. I'm not sure what else it added, or in which priority order.
rgommers
commented
1 year ago Also related to gh-207.
Yea I don't disagree that its confusing for mamba to be doing that. Seems unlikely to change though.
Conda indeed makes a mess of CFLAGS, CXXFLAGS, FFLAGS and LDFLAGS, which is quite annoying. Here is a sample:
$ mamba activate numpy-dev
$ echo $CFLAGS
-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /home/rgommers/mambaforge/envs/numpy-dev/include
$ echo $LDFLAGS
-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,--disable-new-dtags -Wl,--gc-sections -Wl,--allow-shlib-undefined -Wl,-rpath,/home/rgommers/mambaforge/envs/numpy-dev/lib -Wl,-rpath-link,/home/rgommers/mambaforge/envs/numpy-dev/lib -L/home/rgommers/mambaforge/envs/numpy-dev/libThese are the flags used for building packages, but they're injected always rather than only during package build. The rationale of "we want to match what's done during packaging" is useful only to the folks doing packaging, and bad for everyone else. I had to fight to even get things like -fopenmp and -std=c99 removed.
I think what meson and meson-python do is fundamentally correct here, for both the env vars and debug flavors. However, I think we can address this better with some docs on how to get a debug build, with a note on conda envs. Under the How to guides, either as a separate "Using debug builds" or under Use build config settings. Would that work for everyone?
dnicolodi
commented
1 year ago The fact that conda injects environment variables that are known to influence compilation (actually, with the precise intent to influence compilation) is indeed not very nice. Especially because conda users do not seem to be made aware of this. However, there isn't much we can do other than adding documentation. Better documentation is always good. But I would like to try to do not make the documentation too conda-specific.
I think there are two aspects that need to be noted in the documentation: the fact that meson uses some environment variables to define compilation options (and that these variables can be set without the user doing anything), and how to get a debug build.
The place for the first piece of information is probably both in Environment variables and in Default build options.
The second piece of information probably should go in a new section, as @rgommers suggested.
rgommers
commented
1 year ago The second piece of information probably should go in a new section, as @rgommers suggested.
I opened a PR for this: gh-466.
When building pandas, if I specify a command like:
Shared libraries / extensions still get built with
-O2. compile_commands.json confirms that while-O0is passed in, it gets overridden later in the command: