daphne-eu / daphne

DAPHNE: An Open and Extensible System Infrastructure for Integrated Data Analysis Pipelines
Apache License 2.0
67 stars 62 forks source link

redundant compilation of kernels #850

Open philipportner opened 1 month ago

philipportner commented 1 month ago

When recompiling a file or files which are not part of the libAllKernels.so, the kernels do still get compiled again. I'm pretty sure the problem is that execute_process here is done for every call of build.sh, which replaces the kernel files in build/, causing them to have a new timestamp and for ninja to compile them again. See the related ninja issue here.

When running on a system with ccache this does not matter as ccache is able to use the cached compilation even if the timestamps do not match.

As I could not find any option for execute_process to run the command conditionally in the documentation, I think one way to solve this would be to simply check in the genKernelInst.py script if the kernels already exist. Translation units would still be recompiled when the source changes as this is handled by ninja. I don't like this solution as it could cause problems when adding new template instantiations to the kernels.json file.