Issue after 0.4.0 Release

handstandsam commented 1 year ago

I'm monitoring the buildscript classpath, and after bumping to 0.4.0, it correctly detects there was a change. HOWEVER, if I do run the ./gradlew :dependencyGuardBaseline task it does not create a new baseline file and just continues to show the error.

* What went wrong:
Execution failed for task ':buildSrc:dependencyGuard'.
> Dependencies Changed in : for configuration classpath
  - com.dropbox.dependency-guard:com.dropbox.dependency-guard.gradle.plugin:0.3.1
  + com.dropbox.dependency-guard:com.dropbox.dependency-guard.gradle.plugin:0.4.0
  - com.dropbox.dependency-guard:dependency-guard:0.3.1
  + com.dropbox.dependency-guard:dependency-guard:0.4.0

  If this is intentional, re-baseline using ./gradlew :dependencyGuardBaseline
  Or use ./gradlew dependencyGuardBaseline to re-baseline dependencies in entire project.

I then deleted the rm dependencies/classpath.txt file and re-ran the baseline task, but am seeing the same issue. I additionally attempted to add --no-configuration-cache and it still doesn't work as expected. No new baseline file is created, and the same error above is shown.

handstandsam commented 1 year ago

cc: @qwert2603

handstandsam commented 1 year ago

I'm going to try and look into it now as it won't allow us to upgrade, and will block others potentially if they update.

I had issues resolving the snapshot in my large project config, and thought it was fine in the testing I did in a smaller project, but looks like something is up.

I don't have an open source repro case at this point unfortunately.

handstandsam commented 1 year ago

This is definitely something regarding reading the dependencyGuard {} configuration at the right time.

After manually deleting the file causing issue above, I started running into:

An exception occurred applying plugin request [id: 'com.dropbox.dependency-guard']
> Failed to apply plugin 'com.dropbox.dependency-guard'.
   > Could not create task ':dropbox-sdk-java:dependencyGuard'.
      > Error: No configurations provided to Dependency Guard Plugin.
        Here are some valid configurations you could use.

        dependencyGuard {
          configuration("compileClasspath")
          configuration("runtimeClasspath")
          configuration("testCompileClasspath")
          configuration("testRuntimeClasspath")
        }

handstandsam commented 1 year ago

Good news. It looks like after removing the validation code: https://github.com/dropbox/dependency-guard/blob/2a4d18a33ea5cdbf497016e0975b27c2d46a65aa/dependency-guard/src/main/kotlin/com/dropbox/gradle/plugins/dependencyguard/internal/list/DependencyGuardListTask.kt#L164-L172

Now it's working. So, need to fix that.

handstandsam commented 1 year ago

An exception occurred applying plugin request [id: 'com.dropbox.dependency-guard']
> Failed to apply plugin 'com.dropbox.dependency-guard'.
   > Could not create task ':dropbox-sdk-java:dependencyGuard'.
      > Error: No configurations provided to Dependency Guard Plugin.
        Here are some valid configurations you could use.

        dependencyGuard {
          configuration("compileClasspath")
          configuration("runtimeClasspath")
          configuration("testCompileClasspath")
          configuration("testRuntimeClasspath")
        }

Issues seem to be because configuration is not available yet. Therefore configuration can't be validated and tasks can't be created based on config yet. 🤔

handstandsam commented 1 year ago

Some debug output, but the only picked up configurations at that point in time are for the classpath configuration:

monitoredConfigurations: [configurationName: classpath
artifacts: true
modules: false
tree: false
allowedFilter: (kotlin.String) -> kotlin.Boolean
baselineMap: (kotlin.String) -> kotlin.String
]

handstandsam commented 1 year ago

Worked here: https://github.com/dropbox/dropbox-sdk-java/pull/484 Worked here: https://github.com/handstandsam/ShoppingApp/pull/75

But not working in our large monorepo. 🤔

@qwert2603 - Let me know how it works out in your monorepo.

qwert2603 commented 1 year ago

@handstandsam Hi!

I've checked version 0.4.0 on the large repo and everything works well.

qwert2603 commented 1 year ago

Based on the error output, the exception is thrown by ConfigurationValidators.requirePluginConfig and in this method isForRootProject was false. That's strange, because for the buildscript classpath isForRootProject must be true:

https://github.com/dropbox/dependency-guard/blob/2a4d18a33ea5cdbf497016e0975b27c2d46a65aa/dependency-guard/src/main/kotlin/com/dropbox/gradle/plugins/dependencyguard/internal/list/DependencyGuardListTask.kt#L164-L168

Can you provide the dependencyGuard plugin configuration for the buildscript classpath and describe the gradle structure of the root project, please?

qwert2603 commented 1 year ago

Another error is described in #68.

I think, tasks configuration validation may be moved back to the task's action. Also the validation code might be refactored to make it simpler. Now there are two checks in ConfigurationValidators, but they are very similar.

handstandsam commented 1 year ago

@SimonMarquis here is the new ticket to track in. Tagging you so you'll get updates.

qwert2603 commented 1 year ago

Looks like the configuration validation can't be easily moved to the task's execution phase, because method ConfigurationValidators.validateConfigurationsAreAvailable uses Project.configurations field to access availableClasspathConfigs, but Project is not available in the task's execution phase when configuration cache is enabled.

qwert2603 commented 1 year ago

@handstandsam after looking again on the error I've noticed that error happens in non-root module:

Could not create task ':dropbox-sdk-java:dependencyGuard'

Are there any custom configurations created for that module? And is dependencyGuard {...} block goes after the configurations creation on the build.gradle file?

qwert2603 commented 1 year ago

Furthermore, such error is thrown is case when monitoredConfigurations are empty or blank.

handstandsam commented 1 year ago

TL;DR - I think using ./gradlew dependencyGuardBaseline --rerun-tasks fixed it?

In my case, it looks like it works now on my huge project. I bumped from 0.3.2 -> 0.4.0 and was seeing the problems mentioned above.

1) I was monitoring the buildSrc classpath as well. I had to go to the buildSrc module itself, and run ./gradlew dependencyGuardBaseline in that gradle project explicitly. Running the gradle task from the main project threw the error, but wouldn't let me actually re-baseline.

2) I went back to the main Gradle project and ran ./gradlew dependencyGuardBaseline --rerun-tasks - I think this --rerun-tasks is the thing that "fixed" it.

Looking good now in this project. 🤞

Note: I think the other error mentioned needs to still be looked at as it is possibly different.

... Maybe, CI just failed, even with adding --rerun-tasks 🤔

handstandsam commented 1 year ago

Interesting. The task is working when I run ./gradlew dependencyGuard, but not when I run :myapp:dependencyGuard (confirmed locally).

* What went wrong:
A problem occurred evaluating project ':myapp'.
> Failed to apply plugin 'com.dropbox.dependency-guard'.
   > Could not create task ':myapp:dependencyGuard'.
      > Error: No configurations provided to Dependency Guard Plugin.
        Here are some valid configurations you could use.

        dependencyGuard {
        }

Ahh, that is because when I run it, it's only running it for my buildscript classpath. Here is the output confirming:

Dependency Guard baseline created for : for configuration classpath.
File: file:///myfolder/dependencies/classpath.txt

It does Not run for :myapp:dependencyGuard though at that time.

handstandsam commented 1 year ago

I removed my config for monitoring the classpath buildscript for :and now I'm seeing the error above saying there are not configurations provided (which is not true) consistently.

handstandsam commented 1 year ago

I just used Gradle's dependencySubstitution via includeBuild in my large project's settings.gradle file and am getting the same error. When I remove filtering of the configurations (which filter out everything that doesn't end with Classpath),

https://github.com/dropbox/dependency-guard/blob/f875dee735854c2005f54c31b7032c31ad4fc08e/dependency-guard/src/main/kotlin/com/dropbox/gradle/plugins/dependencyguard/internal/ConfigurationValidators.kt#L38-L39

https://github.com/dropbox/dependency-guard/blob/f875dee735854c2005f54c31b7032c31ad4fc08e/dependency-guard/src/main/kotlin/com/dropbox/gradle/plugins/dependencyguard/internal/ConfigurationValidators.kt#L50-L58

Removed that filter. and...

* What went wrong:
A problem occurred evaluating project ':myapp'.
> Failed to apply plugin 'com.dropbox.dependency-guard'.
   > Could not create task ':myapp'.
      > Error: No configurations provided to Dependency Guard Plugin.
        Here are some valid configurations you could use.

        dependencyGuard {
          configuration("androidApis")
          configuration("androidJdkImage")
          configuration("androidTestAnnotationProcessor")
          configuration("androidTestApi")
          configuration("androidTestApiDependenciesMetadata")
          configuration("androidTestCompileOnly")
          configuration("androidTestCompileOnlyDependenciesMetadata")
          configuration("androidTestDebugAnnotationProcessor")
          configuration("androidTestDebugApi")
          configuration("androidTestDebugApiDependenciesMetadata")
          configuration("androidTestDebugCompileOnly")
          configuration("androidTestDebugCompileOnlyDependenciesMetadata")
          configuration("androidTestDebugImplementation")
          configuration("androidTestDebugImplementationDependenciesMetadata")
          configuration("androidTestDebugIntransitiveDependenciesMetadata")
          configuration("androidTestDebugRuntimeOnly")
          configuration("androidTestDebugRuntimeOnlyDependenciesMetadata")
          configuration("androidTestDebugWearApp")
          configuration("androidTestImplementation")
          configuration("androidTestImplementationDependenciesMetadata")
          configuration("androidTestIntransitiveDependenciesMetadata")
          configuration("androidTestReleaseAnnotationProcessor")
          configuration("androidTestReleaseApi")
          configuration("androidTestReleaseApiDependenciesMetadata")
          configuration("androidTestReleaseCompileOnly")
          configuration("androidTestReleaseCompileOnlyDependenciesMetadata")
          configuration("androidTestReleaseImplementation")
          configuration("androidTestReleaseImplementationDependenciesMetadata")
          configuration("androidTestReleaseIntransitiveDependenciesMetadata")
          configuration("androidTestReleaseRuntimeOnly")
          configuration("androidTestReleaseRuntimeOnlyDependenciesMetadata")
          configuration("androidTestReleaseWearApp")
          configuration("androidTestRuntimeOnly")
          configuration("androidTestRuntimeOnlyDependenciesMetadata")
          configuration("androidTestUtil")
          configuration("androidTestWearApp")
          configuration("annotationProcessor")
          configuration("api")
          configuration("apiDependenciesMetadata")
          configuration("archives")
          configuration("compileOnly")
          configuration("compileOnlyDependenciesMetadata")
          configuration("coreLibraryDesugaring")
          configuration("debugAnnotationProcessor")
          configuration("debugApi")
          configuration("debugApiDependenciesMetadata")
          configuration("debugCompileOnly")
          configuration("debugCompileOnlyDependenciesMetadata")
          configuration("debugImplementation")
          configuration("debugImplementationDependenciesMetadata")
          configuration("debugIntransitiveDependenciesMetadata")
          configuration("debugRuntimeOnly")
          configuration("debugRuntimeOnlyDependenciesMetadata")
          configuration("debugWearApp")
          configuration("default")
          configuration("implementation")
          configuration("implementationDependenciesMetadata")
          configuration("intransitiveDependenciesMetadata")
          configuration("kapt")
          configuration("kaptAndroidTest")
          configuration("kaptAndroidTestDebug")
          configuration("kaptAndroidTestRelease")
          configuration("kaptDebug")
          configuration("kaptRelease")
          configuration("kaptTest")
          configuration("kaptTestDebug")
          configuration("kaptTestFixtures")
          configuration("kaptTestFixturesDebug")
          configuration("kaptTestFixturesRelease")
          configuration("kaptTestRelease")
          configuration("kotlinCompilerClasspath")
          configuration("kotlinCompilerPluginClasspath")
          configuration("kotlinKlibCommonizerClasspath")
          configuration("kotlinNativeCompilerPluginClasspath")
          configuration("lintChecks")
          configuration("lintPublish")
          configuration("releaseAnnotationProcessor")
          configuration("releaseApi")
          configuration("releaseApiDependenciesMetadata")
          configuration("releaseCompileOnly")
          configuration("releaseCompileOnlyDependenciesMetadata")
          configuration("releaseImplementation")
          configuration("releaseImplementationDependenciesMetadata")
          configuration("releaseIntransitiveDependenciesMetadata")
          configuration("releaseRuntimeOnly")
          configuration("releaseRuntimeOnlyDependenciesMetadata")
          configuration("releaseWearApp")
          configuration("runtimeOnly")
          configuration("runtimeOnlyDependenciesMetadata")
          configuration("testAnnotationProcessor")
          configuration("testApi")
          configuration("testApiDependenciesMetadata")
          configuration("testCompileOnly")
          configuration("testCompileOnlyDependenciesMetadata")
          configuration("testDebugAnnotationProcessor")
          configuration("testDebugApi")
          configuration("testDebugApiDependenciesMetadata")
          configuration("testDebugCompileOnly")
          configuration("testDebugCompileOnlyDependenciesMetadata")
          configuration("testDebugImplementation")
          configuration("testDebugImplementationDependenciesMetadata")
          configuration("testDebugIntransitiveDependenciesMetadata")
          configuration("testDebugRuntimeOnly")
          configuration("testDebugRuntimeOnlyDependenciesMetadata")
          configuration("testDebugWearApp")
          configuration("testFixturesAnnotationProcessor")
          configuration("testFixturesApi")
          configuration("testFixturesApiDependenciesMetadata")
          configuration("testFixturesCompileOnly")
          configuration("testFixturesCompileOnlyDependenciesMetadata")
          configuration("testFixturesDebugAnnotationProcessor")
          configuration("testFixturesDebugApi")
          configuration("testFixturesDebugApiDependenciesMetadata")
          configuration("testFixturesDebugCompileOnly")
          configuration("testFixturesDebugCompileOnlyDependenciesMetadata")
          configuration("testFixturesDebugImplementation")
          configuration("testFixturesDebugImplementationDependenciesMetadata")
          configuration("testFixturesDebugIntransitiveDependenciesMetadata")
          configuration("testFixturesDebugRuntimeOnly")
          configuration("testFixturesDebugRuntimeOnlyDependenciesMetadata")
          configuration("testFixturesDebugWearApp")
          configuration("testFixturesImplementation")
          configuration("testFixturesImplementationDependenciesMetadata")
          configuration("testFixturesIntransitiveDependenciesMetadata")
          configuration("testFixturesReleaseAnnotationProcessor")
          configuration("testFixturesReleaseApi")
          configuration("testFixturesReleaseApiDependenciesMetadata")
          configuration("testFixturesReleaseCompileOnly")
          configuration("testFixturesReleaseCompileOnlyDependenciesMetadata")
          configuration("testFixturesReleaseImplementation")
          configuration("testFixturesReleaseImplementationDependenciesMetadata")
          configuration("testFixturesReleaseIntransitiveDependenciesMetadata")
          configuration("testFixturesReleaseRuntimeOnly")
          configuration("testFixturesReleaseRuntimeOnlyDependenciesMetadata")
          configuration("testFixturesReleaseWearApp")
          configuration("testFixturesRuntimeOnly")
          configuration("testFixturesRuntimeOnlyDependenciesMetadata")
          configuration("testFixturesWearApp")
          configuration("testImplementation")
          configuration("testImplementationDependenciesMetadata")
          configuration("testIntransitiveDependenciesMetadata")
          configuration("testReleaseAnnotationProcessor")
          configuration("testReleaseApi")
          configuration("testReleaseApiDependenciesMetadata")
          configuration("testReleaseCompileOnly")
          configuration("testReleaseCompileOnlyDependenciesMetadata")
          configuration("testReleaseImplementation")
          configuration("testReleaseImplementationDependenciesMetadata")
          configuration("testReleaseIntransitiveDependenciesMetadata")
          configuration("testReleaseRuntimeOnly")
          configuration("testReleaseRuntimeOnlyDependenciesMetadata")
          configuration("testReleaseWearApp")
          configuration("testRuntimeOnly")
          configuration("testRuntimeOnlyDependenciesMetadata")
          configuration("testWearApp")
          configuration("wearApp")
        }

handstandsam commented 1 year ago

Based on the last comment & output, it looks like the problem is that we don't have access to the Classpath type configurations when executing the DependencyGuardListTask https://github.com/dropbox/dependency-guard/blob/f875dee735854c2005f54c31b7032c31ad4fc08e/dependency-guard/src/main/kotlin/com/dropbox/gradle/plugins/dependencyguard/internal/list/DependencyGuardListTask.kt.

The configuration cache documentation Gradle looks like it potentially provides a way to workaround this (or at least calls out some differences when accessing the Classpath during a Task. Documentation: https://docs.gradle.org/current/userguide/configuration_cache.html#config_cache:requirements

handstandsam commented 1 year ago

It looks like these RuntimeClasspath and CompileClasspath configurations are available if we go into an .afterEvaluate block.

Not there:

  target.configurations.forEach {
                println("* ${it.name}")
            }

There:

  target.afterEvaluate {
            configurations.forEach {
                println("* ${it.name}")
            }
        }

handstandsam commented 1 year ago

lol

https://github.com/gradle/gradle/blob/34b052a2dd0520d0586319c599e128c987427ce7/subprojects/diagnostics/src/main/java/org/gradle/api/tasks/diagnostics/DependencyReportTask.java#L25

handstandsam commented 1 year ago

Last guess of the night.

Configuration Cache Documentation says to use Provider<ResolvedComponentResult> instead of ResolvedComponentResult.

Currently

    @get:Input
    abstract val monitoredConfigurationsMap: MapProperty<DependencyGuardConfiguration, ResolvedComponentResult>

Could be:

    @get:Input
    abstract val monitoredConfigurationsMap: MapProperty<DependencyGuardConfiguration, Provider<ResolvedComponentResult>>

Maybe that'll work? @qwert2603 - Let me know if you have any luck. Thanks again!

handstandsam commented 1 year ago

Hooray, after #70, I am testing in my large project where I had the issue, and I am seeing that we now get the correct output when doing validations of configuration names. 🎉

Could not create task ':myapp:dependencyGuard'.
> Error: No configurations provided to Dependency Guard Plugin.
  Here are some valid configurations you could use.

  dependencyGuard {
    configuration("debugAndroidTestCompileClasspath")
    ...
    configuration("debugRuntimeClasspath")
   }

Previously it was only seeing:

   > Could not create task ':myapp:dependencyGuard'.
      > Error: No configurations provided to Dependency Guard Plugin.
        Here are some valid configurations you could use.

        dependencyGuard {
        }

dropbox / dependency-guard

Issue after 0.4.0 Release #67