Occasionally, after republishing a SNAPSHOT C++ library locally, Gradle fails to fully extract the dependency's headers

[javaJake]

Sometimes I will rapidly re-publish a SNAPSHOT version of a C++ library dependency to my local Maven repository (via "publishToMavenLocal") to use new, unmerged changes in a dependent library. Once in a while, Gradle fails to extract all the headers into its native cache directory during the dependent's build.

Expected Behavior

A published local snapshot version of a dependency library is fully extracted in the native cache directory, and included in the dependent's build.

Current Behavior

The .gradle cache directory will either be missing or only have partially extracted headers for the dependency. The headers zip in the .m2 always looks complete, at least as far as I've observed.

If the .gradle cache directory does have a partially extracted directory, the options.txt will have an appropriate include. Otherwise, the include will be missing, but Gradle will not report an error about a missing dependency.

Either way, the compiler will report missing headers and the build will fail.

Workaround

1. Stop all running Gradle daemons ("./gradlew --stop")
2. Delete the entire .gradle directory in both the home and project directories. So far I haven't figured out which files need to be deleted to "reset" Gradle's view of its native cache.
3. Re-publish the dependency.
4. Re-run the dependent's build.

Context

This has been an issue since 4.10.1, though possibly earlier. The issue remains through 5.0.

Steps to Reproduce (for bugs)

Unfortunately, the issue is not easy to reproduce. It only happens when I least expect it.

1. Dependency: make a change to a .h file, and "./gradlew publishToMavenLocal"
2. Dependent: Run "./gradlew assemble"
3. If it worked, rinse and repeat until it doesn't.

Your Environment

Windows 10 64-bit. Gradle 4.10.1 via wrapper. Gradle C++ build using the modern DSL.

Since the project is closed source I am not able to immediately release example snippets. If I develop something on my own that can reproduce the issue I will publish that here.

[big-guy]

Sometimes I will rapidly re-publish a SNAPSHOT version of a C++ library dependency to my local Maven repository (via "publishToMavenLocal") to use new, unmerged changes in a dependent library. Once in a while, Gradle fails to extract all the headers into its native cache directory during the dependent's build.

You don't need to do this. Use composite builds to glue together independent builds. You don't necessarily have to add the build in your settings.gradle, you can do it sort of "ad hoc" by using --include-build path/to/other/build.

This avoids the publishing and extracting steps.

The .gradle cache directory will either be missing or only have partially extracted headers for the dependency. The headers zip in the .m2 always looks complete, at least as far as I've observed.

I suspect there was something dodgy in the way we extracted these headers. We replaced the way we did this in 5.3, could you give that a try?

[javaJake]

I'll retest with 5.3.

As for the composite build concept, I swear I looked at this before, but I will try again if I have the opportunity.

[lacasseio]

These optimizations would be best done directly inside our dependency engine. If you can provide a sample using the latest Gradle version to show this unnecessary decompression, we can work on optimization. Open the issue directly on https://github.com/gradle/gradle.