Header files in build directory incompatible to compiled library

[hsandmeyer]

When building quda, there are a lot of defines that are generated by cmake. These defines may change objects in the header files. However, the include directory in the build directory is a simple copy of the include directory in the source folder. When instantiating quda objects outside the compile process of quda, these defines are not known to the compiler. This leads to incompatible compiled versions of quda objecs. For example, have a look at USE_TEXTURE_OBJECTS. This is only defined if if __COMPUTE_CAPABILITY__ and CUDA_VERSION are defined. This is why a sizeof(cudaGaugefield) is 936 inside of the quda lib and outside of quda we get sizeof(cudaGaugefield) = 904. So when building quda, one has to make sure that all defines that have been used for compiling are also vaild in the include directory of the build directory.

[mathiaswagner]

First workaround here is to have cmake configure a header with all the necessary includes. Applications using quda headers then need to make sure that this is included before any other quda header.

In a further step we could also move all the compile time definitions into generated header files and make sure these are included in headers / source files as necessary.

[mathiaswagner]

I will script something together. Which defines do we currently need to make sure to have the headers behave properly?

• [ ] __COMPUTE_CAPABILITY__
• [ ] CUDA_VERSION
• [ ] GPU_COMMS
• [ ] QMP_COMMS
• [ ] MULTI_GPU
• [ ] PTHREADS
• [ ] INTERFACE_NVTX
• [ ] DIRECT_ACCESS_BLAS
• [ ] HAVE_QIO
• [ ] ARPACK_LIB @hsandmeyer Can you please add to the list ?

[rbabich]

Applications using quda headers then need to make sure that this is included before any other quda header.

It used to be that quda.h was the only header that an application was expected to include. Is that no longer the case?

Shouldn't the generated header just be included at the beginning of quda.h?

[maddyscientist]

@rbabich Due to intense customer demand 😄 , we need to introduce an interface that uses the underlying C++ types in QUDA allowing external application to manipulate the underlying structures. That's what makes this different from the usual use of the C interface in quda.h.

[rbabich]

I see.

Rather than generate a header, another option is to provide a "quda-config" executable in the build directory that can be used to query the CFLAGS used during the build, as well as library dependencies, etc.

For an example, see https://github.com/usqcd-software/qmp/blob/master/bin/qmp-config.in (or the generated "qmp-config", if you happen to have QMP installed). This approach is used by most (all?) of the various USQCD packages: QMP, QIO, QDP/C, etc.

[mathiaswagner]

@rbabich I know the USQCD way. But I don't see the benefit here. The header only affects the preprocessor defines and once quda is configured and compiled these defines are unique for the build and don't need to be exposed to the user. (They are not really hidden in the configured header).

When it comes to making sure all other compile flags are set correctly (which can also be handled by the USQCD approach) I guess this is a related but different story. (cmake installation and cmake targets that can be imported are probably one way to go here).

So I think we can still do the USQCD approach even if we hide all the preprocessor defines in a configured header.

[mathiaswagner]

Picking up here again. It looks like some simplification will come once quda 0.9 is out. We will drop Fermi and this will remove a lot of #ifdefs