Old output is reused unless I clear all caches

[oakkitten]

I tried using this lovely plugin in a project I'm working on, and I ran into a problem. It works, but only once after cleaning the project (./gradlew clean) and removing build cache (.gradle/caches/build-cache-1). This happens 100% of the time. For some reason the tasks that depend on this plugin's tasks appear UP-TO-DATE. This just might be a problem with the project, but I have no idea how to even begin tackling it.

Here's the commit where I added the plugin. (Note that while the project is easy to build it needs some 5GB+ of dependencies.) Below is the output of a build which I ran after changing a single sting that has ${} in it. Windows 10 here, if that matters, and Gradle 7.5.1.

Executing tasks: [:AnkiDroid:assembleAmazonDebug] in project C:\Users\oakkitten\StudioProjects\Anki-Android

> Task :api:preBuild UP-TO-DATE
> Task :api:preDebugBuild UP-TO-DATE
> Task :api:compileDebugAidl NO-SOURCE
> Task :api:packageDebugRenderscript NO-SOURCE
> Task :api:writeDebugAarMetadata UP-TO-DATE
> Task :api:compileDebugRenderscript NO-SOURCE
> Task :api:generateDebugResValues UP-TO-DATE
> Task :api:generateDebugResources UP-TO-DATE
> Task :api:packageDebugResources UP-TO-DATE
> Task :api:generateDebugBuildConfig UP-TO-DATE
> Task :api:parseDebugLocalResources UP-TO-DATE
> Task :api:processDebugManifest UP-TO-DATE
> Task :AnkiDroid:createAmazonDebugVariantModel UP-TO-DATE
> Task :AnkiDroid:installGitHook UP-TO-DATE
> Task :AnkiDroid:preBuild UP-TO-DATE
> Task :AnkiDroid:preAmazonDebugBuild UP-TO-DATE
> Task :AnkiDroid:mergeAmazonDebugNativeDebugMetadata NO-SOURCE
> Task :AnkiDroid:compileAmazonDebugAidl NO-SOURCE
> Task :AnkiDroid:compileAmazonDebugRenderscript NO-SOURCE
> Task :AnkiDroid:generateAmazonDebugBuildConfig UP-TO-DATE
> Task :api:generateDebugRFile UP-TO-DATE
> Task :api:compileDebugKotlin UP-TO-DATE
> Task :api:javaPreCompileDebug UP-TO-DATE
> Task :api:compileDebugJavaWithJavac UP-TO-DATE
> Task :api:bundleLibRuntimeToJarDebug UP-TO-DATE
> Task :api:extractDeepLinksDebug UP-TO-DATE
> Task :api:compileDebugLibraryResources UP-TO-DATE
> Task :api:bundleLibCompileToJarDebug UP-TO-DATE
> Task :api:mergeDebugShaders UP-TO-DATE
> Task :api:compileDebugShaders NO-SOURCE
> Task :api:generateDebugAssets UP-TO-DATE
> Task :api:packageDebugAssets UP-TO-DATE
> Task :api:processDebugJavaRes NO-SOURCE
> Task :api:bundleLibResDebug UP-TO-DATE
> Task :api:bundleLibRuntimeToDirDebug UP-TO-DATE
> Task :api:mergeDebugJniLibFolders UP-TO-DATE
> Task :api:mergeDebugNativeLibs NO-SOURCE
> Task :api:copyDebugJniLibsProjectOnly UP-TO-DATE
> Task :AnkiDroid:checkAmazonDebugAarMetadata UP-TO-DATE
> Task :AnkiDroid:generateAmazonDebugResValues UP-TO-DATE
> Task :AnkiDroid:mapAmazonDebugSourceSetPaths UP-TO-DATE
> Task :AnkiDroid:generateAmazonDebugResources UP-TO-DATE
> Task :AnkiDroid:commonStringAmazonDebugResourceLocator
> Task :AnkiDroid:templateStringAmazonDebugResourceLocator
> Task :AnkiDroid:templatesAmazonDebugIdentifier
> Task :AnkiDroid:gatherAmazonDebugStringTemplates
> Task :AnkiDroid:resolveAmazonDebugPlaceholders
> Task :AnkiDroid:mergeAmazonDebugResources
> Task :AnkiDroid:createAmazonDebugCompatibleScreenManifests UP-TO-DATE

> Task :AnkiDroid:extractDeepLinksAmazonDebug
Execution optimizations have been disabled for task ':AnkiDroid:extractDeepLinksAmazonDebug' to ensure correctness due to the following reasons:
  - Gradle detected a problem with the following location: 'C:\Users\oakkitten\StudioProjects\Anki-Android\AnkiDroid\build\generated\resolved\amazonDebug\navigation'. Reason: Task ':AnkiDroid:extractDeepLinksAmazonDebug' uses this output of task ':AnkiDroid:resolveAmazonDebugPlaceholders' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.5.1/userguide/validation_problems.html#implicit_dependency for more details about this problem.

> Task :AnkiDroid:processAmazonDebugMainManifest UP-TO-DATE
> Task :AnkiDroid:processAmazonDebugManifest UP-TO-DATE
> Task :AnkiDroid:processAmazonDebugManifestForPackage UP-TO-DATE
> Task :AnkiDroid:processAmazonDebugResources UP-TO-DATE
> Task :AnkiDroid:compileAmazonDebugKotlin UP-TO-DATE
> Task :AnkiDroid:javaPreCompileAmazonDebug UP-TO-DATE
> Task :AnkiDroid:compileAmazonDebugJavaWithJavac UP-TO-DATE
> Task :AnkiDroid:mergeAmazonDebugShaders UP-TO-DATE
> Task :AnkiDroid:compileAmazonDebugShaders NO-SOURCE
> Task :AnkiDroid:generateAmazonDebugAssets UP-TO-DATE
> Task :AnkiDroid:mergeAmazonDebugAssets UP-TO-DATE
> Task :AnkiDroid:compressAmazonDebugAssets UP-TO-DATE
> Task :AnkiDroid:l8DexDesugarLibAmazonDebug UP-TO-DATE
> Task :AnkiDroid:generateAmazonDebugJacocoPropertiesFile UP-TO-DATE
> Task :AnkiDroid:processAmazonDebugJavaRes UP-TO-DATE
> Task :AnkiDroid:mergeAmazonDebugJavaResource UP-TO-DATE
> Task :AnkiDroid:checkAmazonDebugDuplicateClasses UP-TO-DATE
> Task :AnkiDroid:desugarAmazonDebugFileDependencies UP-TO-DATE
> Task :AnkiDroid:mergeExtDexAmazonDebug UP-TO-DATE
> Task :AnkiDroid:mergeLibDexAmazonDebug UP-TO-DATE
> Task :AnkiDroid:jacocoAmazonDebug UP-TO-DATE
> Task :AnkiDroid:dexBuilderAmazonDebug UP-TO-DATE
> Task :AnkiDroid:mergeProjectDexAmazonDebug UP-TO-DATE
> Task :AnkiDroid:mergeAmazonDebugJniLibFolders UP-TO-DATE
> Task :AnkiDroid:mergeAmazonDebugNativeLibs UP-TO-DATE
> Task :AnkiDroid:stripAmazonDebugDebugSymbols UP-TO-DATE
> Task :AnkiDroid:validateSigningAmazonDebug UP-TO-DATE
> Task :AnkiDroid:writeAmazonDebugAppMetadata UP-TO-DATE
> Task :AnkiDroid:writeAmazonDebugSigningConfigVersions UP-TO-DATE
> Task :AnkiDroid:packageAmazonDebug UP-TO-DATE
> Task :AnkiDroid:createAmazonDebugApkListingFileRedirect UP-TO-DATE
> Task :AnkiDroid:assembleAmazonDebug UP-TO-DATE

Deprecated Gradle features were used in this build, making it incompatible with Gradle 8.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/7.5.1/userguide/command_line_interface.html#sec:command_line_warnings

Execution optimizations have been disabled for 1 invalid unit(s) of work during this build to ensure correctness.
Please consult deprecation warnings for more details.

BUILD SUCCESSFUL in 16s
63 actionable tasks: 7 executed, 56 up-to-date

Build Analyzer results available

[LikeTheSalad]

Hi @oakkitten - I remember a similar issue that was only happening on Windows machines due to the '/' '\' file path separator differences. It should be fixed now, however for some reason you seem to be getting the same issue.

In order to know what could be happening, I'm going to need the --info console output for when you build your project for both cases, when it works (after a clean) and when it doesn't. In order to get this console output, you need to build your project this way: ./gradlew :AnkiDroid:assembleAmazonDebug --info

Please let me know if any questions.

[oakkitten]

Sure, here's the logs. Just say if you need anything else.

• 1-clean-build.txt — output of ./gradlew :AnkiDroid:assembleAmazonDebug --info after doing ./gradlew clean, removing ~/.gradle/caches/build-cache-1 and killing the daemon
• 2-build-right-after-clean-build.txt — output of the same command ran just after
• 3-build-with-one-string-changed.txt — output of the same command ran just after changing one string with a ${}.

[LikeTheSalad]

Hi @oakkitten - Thanks for the info. I was checking it out and I noticed that it all seems to be working well in terms of normal Stem execution, I can tell this because on the 3rd file, line 1437, it reads: Task ':AnkiDroid:resolveAmazonDebugPlaceholders' is not up-to-date because: followed by the files changed after you modified a string with ${}, meaning that, the resolving process did run and the overall output of Stem should be updated after that.

Now, I did find something that could be the problem you're facing, in the 1st file on line 12343, I see this line: - Gradle detected a problem with the following location: 'C:\Users\s\StudioProjects\Anki-Android\AnkiDroid\build\generated\resolved\amazonDebug'. Reason: Task ':AnkiDroid:extractDeepLinksAmazonDebug' uses this output of task ':AnkiDroid:resolveAmazonDebugPlaceholders' without declaring an explicit or implicit dependency. This can lead to incorrect results being produced, depending on what order the tasks are executed. Please refer to https://docs.gradle.org/7.5.1/userguide/validation_problems.html#implicit_dependency for more details about this problem. - Based on that, it seems like there's a task in your project named extractDeepLinksAmazonDebug that is gathering Stem's output without explicitly declaring them as its inputs. I recommend you to follow the link above to see what are your options to solve it. I think it should simply be a matter of adding the dependency on Stem's task like so: extractDeepLinksAmazonDebug.dependsOn("resolveAmazonDebugPlaceholders") in your build.gradle file.

I hope it helps.

[oakkitten]

I tried adding the following into the app gradle file

afterEvaluate {
    tasks.getByPath(':AnkiDroid:extractDeepLinksAmazonDebug').dependsOn(':AnkiDroid:resolveAmazonDebugPlaceholders')
}

This does get rid of the warning, but otherwise does not help. 😔

I also ran the task tree tool. This is probably obvious for anyone who made a gradle plugin, but it seems that processAmazonDebugResources depends on mergeAmazonDebugResources and that depends on resolveAmazonDebugPlaceholders. Somehow even though the resolve task does run and produces a correct resolved.xml file, and also merge task runs, the process task seems to remain UP-TO-DATE.

Task tree

``` ./gradlew :AnkiDroid:assembleAmazonDebug taskTree > Task :AnkiDroid:taskTree ------------------------------------------------------------ Project ':AnkiDroid' ------------------------------------------------------------ :AnkiDroid:assembleAmazonDebug +--- :AnkiDroid:mergeAmazonDebugNativeDebugMetadata | \--- :AnkiDroid:preAmazonDebugBuild | \--- :AnkiDroid:preBuild | \--- :AnkiDroid:installGitHook \--- :AnkiDroid:packageAmazonDebug +--- :AnkiDroid:compileAmazonDebugJavaWithJavac | +--- :AnkiDroid:compileAmazonDebugAidl | | +--- :AnkiDroid:preAmazonDebugBuild * | | \--- :api:compileDebugAidl | | \--- :api:preDebugBuild | | \--- :api:preBuild | +--- :AnkiDroid:compileAmazonDebugKotlin | | +--- :AnkiDroid:compileAmazonDebugAidl * | | +--- :AnkiDroid:compileAmazonDebugRenderscript | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | \--- :api:packageDebugRenderscript | | | \--- :api:preDebugBuild * | | +--- :AnkiDroid:generateAmazonDebugBuildConfig | | | \--- :AnkiDroid:preAmazonDebugBuild * | | +--- :AnkiDroid:processAmazonDebugResources | | | +--- :AnkiDroid:checkAmazonDebugAarMetadata | | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | | \--- :api:writeDebugAarMetadata | | | | \--- :api:preDebugBuild * | | | +--- :AnkiDroid:mapAmazonDebugSourceSetPaths | | | | +--- :AnkiDroid:compileAmazonDebugRenderscript * | | | | +--- :AnkiDroid:generateAmazonDebugResValues | | | | | \--- :AnkiDroid:preAmazonDebugBuild * | | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | | \--- :api:packageDebugResources | | | | +--- :api:compileDebugRenderscript | | | | | \--- :api:preDebugBuild * | | | | +--- :api:generateDebugResValues | | | | | \--- :api:preDebugBuild * | | | | +--- :api:generateDebugResources | | | | | +--- :api:compileDebugRenderscript * | | | | | \--- :api:generateDebugResValues * | | | | \--- :api:preDebugBuild * | | | +--- :AnkiDroid:mergeAmazonDebugResources | | | | +--- :AnkiDroid:compileAmazonDebugRenderscript * | | | | +--- :AnkiDroid:generateAmazonDebugResValues * | | | | +--- :AnkiDroid:generateAmazonDebugResources | | | | | +--- :AnkiDroid:compileAmazonDebugRenderscript * | | | | | \--- :AnkiDroid:generateAmazonDebugResValues * | | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | | +--- :AnkiDroid:resolveAmazonDebugPlaceholders | | | | | \--- :AnkiDroid:gatherAmazonDebugStringTemplates | | | | | +--- :AnkiDroid:commonStringAmazonDebugResourceLocator | | | | | | +--- :AnkiDroid:generateAmazonDebugResValues * | | | | | | +--- :api:bundleLibRuntimeToJarDebug | | | | | | | +--- :api:compileDebugJavaWithJavac | | | | | | | | +--- :api:compileDebugAidl * | | | | | | | | +--- :api:compileDebugKotlin | | | | | | | | | +--- :api:compileDebugAidl * | | | | | | | | | +--- :api:compileDebugRenderscript * | | | | | | | | | +--- :api:generateDebugBuildConfig | | | | | | | | | | \--- :api:preDebugBuild * | | | | | | | | | \--- :api:generateDebugRFile | | | | | | | | | +--- :api:parseDebugLocalResources | | | | | | | | | | +--- :api:packageDebugResources * | | | | | | | | | | \--- :api:preDebugBuild * | | | | | | | | | +--- :api:preDebugBuild * | | | | | | | | | \--- :api:processDebugManifest | | | | | | | | | \--- :api:preDebugBuild * | | | | | | | | +--- :api:compileDebugRenderscript * | | | | | | | | +--- :api:generateDebugBuildConfig * | | | | | | | | +--- :api:generateDebugRFile * | | | | | | | | +--- :api:javaPreCompileDebug | | | | | | | | | \--- :api:preDebugBuild * | | | | | | | | \--- :api:preDebugBuild * | | | | | | | +--- :api:compileDebugKotlin * | | | | | | | \--- :api:preDebugBuild * | | | | | | \--- :api:packageDebugResources * | | | | | \--- :AnkiDroid:templatesAmazonDebugIdentifier | | | | | \--- :AnkiDroid:templateStringAmazonDebugResourceLocator | | | | \--- :api:packageDebugResources * | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | +--- :AnkiDroid:processAmazonDebugManifest | | | | +--- :AnkiDroid:createAmazonDebugCompatibleScreenManifests | | | | | \--- :AnkiDroid:preAmazonDebugBuild * | | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | | \--- :AnkiDroid:processAmazonDebugMainManifest | | | | +--- :AnkiDroid:extractDeepLinksAmazonDebug | | | | | +--- :AnkiDroid:compileAmazonDebugRenderscript * | | | | | +--- :AnkiDroid:generateAmazonDebugResValues * | | | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | | | \--- :AnkiDroid:resolveAmazonDebugPlaceholders * | | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | | +--- :api:extractDeepLinksDebug | | | | | +--- :api:compileDebugRenderscript * | | | | | +--- :api:generateDebugResValues * | | | | | \--- :api:preDebugBuild * | | | | \--- :api:processDebugManifest * | | | +--- :AnkiDroid:processAmazonDebugManifestForPackage | | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | | \--- :AnkiDroid:processAmazonDebugManifest * | | | +--- :api:compileDebugLibraryResources | | | | +--- :api:packageDebugResources * | | | | \--- :api:preDebugBuild * | | | \--- :api:generateDebugRFile * | | \--- :api:bundleLibCompileToJarDebug | | +--- :api:compileDebugJavaWithJavac * | | +--- :api:compileDebugKotlin * | | +--- :api:generateDebugRFile * | | \--- :api:preDebugBuild * | +--- :AnkiDroid:compileAmazonDebugRenderscript * | +--- :AnkiDroid:generateAmazonDebugBuildConfig * | +--- :AnkiDroid:javaPreCompileAmazonDebug | | \--- :AnkiDroid:preAmazonDebugBuild * | +--- :AnkiDroid:preAmazonDebugBuild * | +--- :AnkiDroid:processAmazonDebugResources * | \--- :api:bundleLibCompileToJarDebug * +--- :AnkiDroid:compressAmazonDebugAssets | +--- :AnkiDroid:mergeAmazonDebugAssets | | +--- :AnkiDroid:compileAmazonDebugShaders | | | +--- :AnkiDroid:mergeAmazonDebugShaders | | | | \--- :AnkiDroid:preAmazonDebugBuild * | | | \--- :AnkiDroid:preAmazonDebugBuild * | | +--- :AnkiDroid:generateAmazonDebugAssets | | | \--- :AnkiDroid:compileAmazonDebugShaders * | | +--- :AnkiDroid:preAmazonDebugBuild * | | \--- :api:packageDebugAssets | | +--- :api:compileDebugShaders | | | +--- :api:mergeDebugShaders | | | | \--- :api:preDebugBuild * | | | \--- :api:preDebugBuild * | | +--- :api:generateDebugAssets | | | \--- :api:compileDebugShaders * | | \--- :api:preDebugBuild * | \--- :AnkiDroid:preAmazonDebugBuild * +--- :AnkiDroid:l8DexDesugarLibAmazonDebug | \--- :AnkiDroid:preAmazonDebugBuild * +--- :AnkiDroid:mergeAmazonDebugAssets * +--- :AnkiDroid:mergeAmazonDebugJavaResource | +--- :AnkiDroid:compileAmazonDebugJavaWithJavac * | +--- :AnkiDroid:compileAmazonDebugKotlin * | +--- :AnkiDroid:generateAmazonDebugJacocoPropertiesFile | | \--- :AnkiDroid:preAmazonDebugBuild * | +--- :AnkiDroid:preAmazonDebugBuild * | +--- :AnkiDroid:processAmazonDebugJavaRes | | \--- :AnkiDroid:preAmazonDebugBuild * | \--- :api:bundleLibResDebug | +--- :api:compileDebugKotlin * | +--- :api:preDebugBuild * | \--- :api:processDebugJavaRes | \--- :api:preDebugBuild * +--- :AnkiDroid:mergeExtDexAmazonDebug | +--- :AnkiDroid:checkAmazonDebugDuplicateClasses | | \--- :AnkiDroid:preAmazonDebugBuild * | +--- :AnkiDroid:desugarAmazonDebugFileDependencies | | \--- :AnkiDroid:preAmazonDebugBuild * | \--- :AnkiDroid:preAmazonDebugBuild * +--- :AnkiDroid:mergeLibDexAmazonDebug | +--- :AnkiDroid:checkAmazonDebugDuplicateClasses * | \--- :AnkiDroid:preAmazonDebugBuild * +--- :AnkiDroid:mergeProjectDexAmazonDebug | +--- :AnkiDroid:checkAmazonDebugDuplicateClasses * | +--- :AnkiDroid:dexBuilderAmazonDebug | | +--- :AnkiDroid:jacocoAmazonDebug | | | +--- :AnkiDroid:compileAmazonDebugJavaWithJavac * | | | +--- :AnkiDroid:compileAmazonDebugKotlin * | | | +--- :AnkiDroid:preAmazonDebugBuild * | | | \--- :AnkiDroid:processAmazonDebugResources * | | +--- :AnkiDroid:preAmazonDebugBuild * | | +--- :api:bundleLibCompileToJarDebug * | | \--- :api:bundleLibRuntimeToJarDebug * | \--- :AnkiDroid:preAmazonDebugBuild * +--- :AnkiDroid:preAmazonDebugBuild * +--- :AnkiDroid:processAmazonDebugManifestForPackage * +--- :AnkiDroid:processAmazonDebugResources * +--- :AnkiDroid:stripAmazonDebugDebugSymbols | +--- :AnkiDroid:mergeAmazonDebugNativeLibs | | +--- :AnkiDroid:mergeAmazonDebugJniLibFolders | | | \--- :AnkiDroid:preAmazonDebugBuild * | | +--- :AnkiDroid:preAmazonDebugBuild * | | \--- :api:copyDebugJniLibsProjectOnly | | +--- :api:mergeDebugNativeLibs | | | +--- :api:mergeDebugJniLibFolders | | | | \--- :api:preDebugBuild * | | | \--- :api:preDebugBuild * | | \--- :api:preDebugBuild * | \--- :AnkiDroid:preAmazonDebugBuild * +--- :AnkiDroid:validateSigningAmazonDebug | \--- :AnkiDroid:preAmazonDebugBuild * +--- :AnkiDroid:writeAmazonDebugAppMetadata | \--- :AnkiDroid:preAmazonDebugBuild * \--- :AnkiDroid:writeAmazonDebugSigningConfigVersions \--- :AnkiDroid:preAmazonDebugBuild * (*) - subtree omitted (printed previously) Add --repeat to allow printing a subtree of the same task more than once. ```

[LikeTheSalad]

I see, it seems like your task to extract deeplinks might not be properly wired into the Gradle task graph. I'd say the best way to connect tasks is by connecting their outputs/inputs, as explained here: https://docs.gradle.org/current/userguide/lazy_configuration.html#working_with_task_dependencies_in_lazy_properties

Since the overall Stem work is done properly, I'm going to close this ticket.

[oakkitten]

Hm, I suppose that would mean there's a bug in AGP, then? Is this something that we could report to Google?

[LikeTheSalad]

It depends on what your task is supposed to do, if it's gathering all the resources from your app, then I'd recommend taking a look at the AGP API to see if they have a native way of providing them. They used to provide it, however, they've changed their API a lot starting from AGP 7, so I'd recommend you to first take a look at their new APIs and check if there's a 'native' way from the AGP to provide your task with the project's resources, if there isn't, then I guess you could open an issue about it on Google, since they used to provide that option with their old API.

[oakkitten]

It's not our own task, I believe it comes from AGP. This, I guess?

[LikeTheSalad]

Oh! I see, I wasn't aware of it. I'll take a deeper look at that task to see if it's possible to properly connect it to Stem.

[LikeTheSalad]

Hi @oakkitten - I've just released version 2.3.0 to make Stem aware of the "extractDeeplinks" task, so you will no longer see the warnings, also, I've tested it with deeplinks and they keep up-to-date after changes to the templates or their values, even though the "extractDeeplinks" tasks doesn't run again, because what it seems to do is to set deeplinks to strings ID references rather than values, so if the value of a string changes but its ID doesn't, then the extractDeepkinks task doesn't have anything to change. Please let me know if you find any issues related to deeplinks.