Anvil causes Gradle tasks to be non-cacheable

[1gravity]

When analyzing our Gradle builds, we see that the Kotlin compile task cannot be cached by Gradle. The error message always shows for the app module and sometimes also for library modules:

Overlapping outputs: Gradle does not know how file 'build/anvil/devDebug/caches/generated-file-cache.bin' was created (output property '$2'). Task output caching requires exclusive access to output paths to guarantee correctness (i.e. multiple tasks are not allowed to produce output in the same location).

Kind TASK   Path :mobile:compileDevDebugKotlin Type org.jetbrains.kotlin.gradle.tasks.KotlinCompile

Dagger 2.51.1 Anvil 2.5.0-beta09 Kotlin 1.9.23

com.squareup.anvil.trackSourceFiles=true

This is an issue for us since non-cacheable tasks increase the local builds times considerably

[RBusarow]

Thanks for the report.

That warning message means that some other task is declaring that directory, that file, or some other parent directory as its own output. The warning will only happen when both of the tasks are actually configured and in the task graph, which might explain why it only sometimes happens with library modules.

The Anvil plugin creates cache directories that are unique to each build variant (e.g., devDebug), and they're all direct children of $buildDir/anvil, so Anvil won't create overlaps on its own.

To find the other task, you can add an assertion against the output files of other tasks. For example, to find out the culprit for :mobile:compileDevDebugKotlin, you could add this to that project's build.gradle.kts:

val overlapped = generateSequence(
  file("build/anvil/devDebug/caches/generated-file-cache.bin"),
) { it.parentFile }
  .takeWhile { it != rootDir }
  .toList()

tasks.named { it != "compileDevDebugKotlin" }.configureEach {
  doLast {
    val overlaps = overlapped.filter { outputs.files.contains(it) }
    check(overlaps.isEmpty()) {
      "Overlapping output files:\n${overlaps.joinToString("\n")}"
    }
  }
}

After that, if you just run the same task(s) that prompted the warning before, it should tell you where the overlap is coming from.

[1gravity]

Thanks @RBusarow for getting back to me so quickly. I tried this but none of the tasks seem to have an overlap while I still get the overlapping output message in the Gradle build scan. I had to convert the script to Groovy because we haven't converted all of our build files to Kotlin yet but I think this should work:

def overlapped = []
def currentFile = file("build/anvil/devDebug/caches/generated-file-cache.bin")
while (currentFile != rootDir) {
    overlapped.add(currentFile)
    currentFile = currentFile.parentFile
}

tasks.configureEach {
    if (name != "compileDevDebugKotlin") {
        doLast {
            def overlaps = overlapped.findAll { outputs.files.contains(it) }
            if (!overlaps.isEmpty()) {
                throw new GradleException("Overlapping output files:\n${overlaps.join('\n')}")
            }
        }
    }
}

I put in print statements to verify that there's indeed no overlap (there isn't). I am working through past releases of our app to see when this starts happening. I don't see the issue with Anvil 2.4.9 but will keep testing.

[1gravity]

I can say with a high level of certainty that the issue was introduced with a 2.5.0-beta0X version. I don't see it with 2.4.9 and it shows the moment we updated to 2.5.0-beta03 which we did because of https://github.com/square/anvil/issues/693. I compared build times for a one line of code change in the app module and times went from around 1m 30s to 2m 30s with beta03 and are now at 5m 10s with beta09. I took our latest stable version and reverted back to Anvil 2.4.9 and the build times dropped from >5m to 1m 45s for that simple test case I used. I don't know if that's all due to the Overlapping outputs issue but that seems to be a good starting point.

I just tested with 2.5.0-beta10 and still the same issue.

[1gravity]

Duplicate of https://github.com/square/anvil/issues/821 mentioned in https://github.com/square/anvil/issues/693#issuecomment-1885620524?

[1gravity]

After running more tests, I found that three of our Gradle projects show that error when we run all tasks (--rerun-tasks):

What they have in common is that kapt is running the Dagger compiler. I removed it from the two library modules using generateDaggerFactories.set(true) and that eliminates the error from those two library modules. However I cannot disable kapt/Dagger in the app module since we need Dagger to create the components.

My assumption is that there is some interaction between kapt and the Anvil plugin.

[1gravity]

I was able to reproduce the issue in a sample app that I pushed to Github: https://github.com/1gravity/anvil-tests.

When I make a change in the app module and run ./gradlew :app:assembleDebug --scan I can see the error in the Gradle scan: https://scans.gradle.com/s/waavwoxnjlvj4/timeline?cacheability=any_non_cacheable&hide-timeline&kind=task&outcome=success,failed&sort=longest

I would very much appreciate if someone could try this and give me some guidance on what needs to change

[RBusarow]

Thanks for the repro. It seems to be an issue with KAPT. If you run this, the problem goes away:

./gradlew :app:compileDebugKotlin -x :app:kaptDebugKotlin

It's not immediately apparent to me why that's happening. I should have time to dig into it more this evening.

[chrisbanes]

Just +1'ing this. We're seeing the same.

[nwagu]

This is an issue only when trackSourceFiles is enabled, right?

[bmuschko]

I am seeing the same issue. +1