search

nokeedev / gradle-native

The home of anything about Gradle support for natively compiled languages
https://nokee.dev
Apache License 2.0
47 stars 8 forks source link

Core Gradle header dependencies is sensitive to the source ordering #864

Open lacasseio opened 4 months ago

lacasseio commented 4 months ago

Because of this condition in DefaultIncrementalCompileSourceProcessor, the order in which Gradle process the source file will affect what headers are discovered. It boils down to the fact that if Gradle processes a file containing a macro include and discovers more defines for the macro - e.g. BOOST_PP_FILENAME_1 - the result will depend on whether or not the new definitions are found before or after the macro include was resolved. The condition tries to handle include cycles but disregards the case when the cycle has additional "visible macros". This particular case will be problematic if the source ordering is not stable.

We have seen the ordering issues with the source files, which we are still tracking down. For now, the simplest patch seems to simply order the source files before processing them.

Kelly12612 commented 4 months ago

Hi Daniel, we ran into an issue like this on Windows, when the case of the #include didn't match the case of the file itself, this resulted in longer builds/rebuilds and other sorts of issues. You helped us come up with a workaround in "GLM" (the gradle legacy gitub project which is meant to be an example of converting an older project to gradle/Nokee).

lacasseio commented 3 months ago

This issue is a bit different than the one we encountered. This one breaks Gradle cacheability when compiling against complicated preprocessor libraries such as Boost. The issue is twofold: 1) FileTree ordering is not guaranteed and 2) Gradle can/may/will discover a different set of headers between builds because of the unguaranteed ordering.

The first part seems to be inconsistent between operating systems which is why this issue was complicated to diagnose. Officially, the documentation states: "The order of the files in a FileTree is not stable, even on a single computer." In practice, it seems the ordering is stable under some operating system (i.e. on macOS, I was never able to reproduce the issue but a client reproduced it consistently under Linux).

The second part is very flaky. Because of the conditions outlined in the description, if there are additional macro definitions to discover after realizing a macro includes, Gradle can discover a different set of headers. If the FileTree ordering is indeed not stable on your machine but the project isn't that big, Gradle may discover a different set of headers. Finally, if the project is sizable, Gradle will discover a different set of headers.

We validated a fix that ensures stability but will not fix the underlying issue.