BLD: Generate `config.h` header file

[mtsokol]

Hi @HaoZeke,

Here's my attempt to generate config.h file for OpenBLAS project.

With this change I can successfully build it on qgpu Linux machine with commands:

meson setup build --reconfigure --buildtype release
meson compile -C build

Design

The config.h header generation is divided into several steps:

1. First we need to run c_check and f_check scripts. This is done in the setup stage by using run_command(...). Thanks to this we have _config_1.h with the first batch of contents. TODO: ./build/ path is hardcoded. It should use meson-provided path.
2. The rest of steps happen in meson compile stage. Then we compile getarch.
3. Next we define a custom target _config_2.h that is the output of ./getarch 1
4. Then we merge _config_1.h and _config_2.h into _config_for_getarch_2nd.h into one custom target.
5. Compiling getarch_2nd with _config_for_getarch_2nd_h dependency.
6. The last piece of header input that we need is _config_3.h which is a custom target from calling ./getarch_2nd 1.
7. The last step is merging _config_for_getarch_2nd.h and _config_3.h into a final config.h.

Then I add config_h as an input in all static_library(...) which require config.h header. I followed the pattern described in https://mesonbuild.com/Generating-sources.html#generating-headers to implement it. Ideally we could remove the intermediate _config...h but it's a detail.

Questions

• WDYT about this approach? The generated config.h is exactly as make one on my Linux machine.
• In getarch_2nd.c I had to add an #ifndef to include my merged pre-header as meson doesn't allow targets with the same name. It's behind a BUILD_WITH_MESON that I set in meson.build. Does it make sense?
• I had to comment out three lines in lapack-netlib/meson.build as add_project_arguments can't happen after the first custom_target (I think) but it only sets a warning flag.

[HaoZeke]

Excellent! I like this, its more concise and elegant than what I had in mind with the subprojects, some nits aside this would be the way forward.

[mtsokol]

Addressed all comments!

[HaoZeke]

Almost there, oddly enough my getarch 1 is slightly different with this setup than make

0a1
> // ./c_check Makefile.conf config.h gcc
6a8
> // ./f_check Makefile.conf config.h gfortran
9c11
< 
---
> // ./getarch 1
11c13
< #define L1_CODE_SIZE 32768
---
> #define L1_CODE_SIZE 65536
14,15c16,17
< #define L1_DATA_SIZE 49152
< #define L1_DATA_ASSOCIATIVE 12
---
> #define L1_DATA_SIZE 32768
> #define L1_DATA_ASSOCIATIVE 8
17,18c19,20
< #define L2_SIZE 1310720
< #define L2_ASSOCIATIVE 7
---
> #define L2_SIZE 2097152
> #define L2_ASSOCIATIVE 16
37,38c39
< 
< 
---
> // From getarch_2nd
44d44
< 

I'll try to see why soonish.

[mtsokol]

@HaoZeke I think it might be a missing flag when building getarch binary (I think this output is from getach 1 call)

getarch in make is build with:

getarch : getarch.c cpuid.S dummy $(CPUIDEMU)
    avx512=$$(./c_check$(SCRIPTSUFFIX) - - "$(CC)" $(TARGET_FLAGS) $(CFLAGS) | grep NO_AVX512); \
    rv64gv=$$(./c_check$(SCRIPTSUFFIX) - - "$(CC)" $(TARGET_FLAGS) $(CFLAGS) | grep NO_RV64GV); \
    $(HOSTCC) $(HOST_CFLAGS) $(EXFLAGS) $${avx512:+-D$${avx512}} $${rv64gv:+-D$${rv64gv}} -o $(@F) getarch.c cpuid.S $(CPUIDEMU)

where with meson.build it's a simple executable('getarch', ['getarch.c', 'cpuid.S'])

Maybe a missing flag?

[HaoZeke]

With make -n -p I get:

'/OpenBLAS' avx512=$$(./c_check - - "/micromamba/envs/numpy-dev/bin/x86_64-conda-linux-gnu-cc"  -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /micromamba/envs/numpy-dev/include | grep NO_AVX512); \ rv64gv=$$(./c_check - - "/micromamba/envs/numpy-dev/bin/x86_64-conda-linux-gnu-cc"  -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -ffunction-sections -pipe -isystem /micromamba/envs/numpy-dev/include | grep NO_RV64GV); \ /micromamba/envs/numpy-dev/bin/x86_64-conda-linux-gnu-cc -march=native -DGEMM_MULTITHREAD_THRESHOLD=4 -DNO_PARALLEL_MAKE=0  $${avx512:+-D$${avx512}} $${rv64gv:+-D$${rv64gv}} -o getarch getarch.c cpuid.S  ./c_check Makefile.conf config.h

However both avx512 and rv64gv are empty with this invocation, and even without any additional flags:

gcc getarch.c cpuid.S -o fakegetarch
./c_check Makefile.conf blah gcc
./f_check Makefile.conf blah gfortran
./fakegetarch 1 >> blah
./getarch_2nd 1 >> blah
diff blah wrking_conf.h

Gives the correct set of #defines.

0a1
> // ./c_check Makefile.conf config.h gcc
6a8
> // ./f_check Makefile.conf config.h gfortran
8a11
> // ./getarch 1
35a39
> // From getarch_2nd

More clearly with $(warning) calls, cflags is -march=native -DGEMM_MULTITHREAD_THRESHOLD=4 -DNO_PARALLEL_MAKE=0.

Will look into the meson build more.

[HaoZeke]

Weirdly works for me now

[HaoZeke]

Thanks @mtsokol 🚀