search

leandroBorgesFerreira / dag-command

Affected gradle modules by branch
Apache License 2.0
45 stars 11 forks source link

dag-command stuck #29

Closed hellosagar closed 6 months ago

hellosagar commented 1 year ago

Stuck here forever Screenshot 2023-09-27 at 2 48 37 PM

hellosagar commented 1 year ago

on commenting this line my app build.gradle.kts it started working, but why 🤔 I need this dependency there

// baselineProfile(project(":benchmark"))
leandroBorgesFerreira commented 1 year ago

Hello hellosagar.

Which version of the library are you using? I remember fixing a dependency cycle in some past version. Are you in 1.6.0?

Could you also share your configuration of the dag-plugin command?

hellosagar commented 1 year ago

Hi @leandroBorgesFerreira , Thanks for the response.

Currently using version 1.5.3 Config

project.extensions.getByType(com.github.leandroborgesferreira.dagcommand.extension.CommandExtension::class.java)
  .apply {
    filter = "all"
    defaultBranch = "origin/develop"
    outputType = "json"
    printModulesInfo = true
  }

On using the version 1.6.0, build failed with this error message

> Task :dag-command FAILED
--- Config ---
Filter: all
Default branch: origin/develop
Output type: json
Output path: /Users/sagarkhurana/androidsecond/android/build
--------------

Writing adjacency list... Done

Writing edges list... Done

Build stages... Done

Graph information:
Nodes count: 44
Edges count: 421
Build stages: 17
Build coefficient: 10.772727272727273
Changed modules: benchmark, booking_assistant, checkout, config, destination, domain, app, reviewsubmission

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dag-command'.
> java.lang.StackOverflowError (no error message)

* Try:
> Run with --stacktrace option to get the stack trace.
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/8.0/userguide/command_line_interface.html#sec:command_line_warnings
leandroBorgesFerreira commented 1 year ago

Could you please update to 1.6.0 and tell me if the problem is still there?

hellosagar commented 1 year ago

On using the version 1.6.0, problem still persists

> Task :dag-command FAILED
--- Config ---
Filter: all
Default branch: origin/develop
Output type: json
Output path: /Users/sagarkhurana/androidsecond/android/build
--------------

Writing adjacency list... Done

Writing edges list... Done

Build stages... Done

Graph information:
Nodes count: 44
Edges count: 421
Build stages: 17
Build coefficient: 10.772727272727273
Changed modules: activitycontent, booking_additional_information, bookings, checkout, compass, customviews, destination, domain, app, resources, sdui-core, search

FAILURE: Build failed with an exception.

* What went wrong:
Execution failed for task ':dag-command'.
> java.lang.StackOverflowError (no error message)

* Try:
> Run with --info or --debug option to get more log output.
> Run with --scan to get full insights.

* Exception is:
org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':dag-command'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.lambda$executeIfValid$1(ExecuteActionsTaskExecuter.java:149)
        at org.gradle.internal.Try$Failure.ifSuccessfulOrElse(Try.java:282)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:147)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:135)
        at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46)
        at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:74)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52)
        at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:42)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:338)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:325)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:318)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:304)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:463)
        at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:380)
        at org.gradle.execution.plan.DefaultPlanExecutor.process(DefaultPlanExecutor.java:116)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph.executeWithServices(DefaultTaskExecutionGraph.java:136)
        at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph.execute(DefaultTaskExecutionGraph.java:121)
        at org.gradle.execution.SelectedTaskExecutionAction.execute(SelectedTaskExecutionAction.java:35)
        at org.gradle.execution.DryRunBuildExecutionAction.execute(DryRunBuildExecutionAction.java:51)
        at org.gradle.execution.BuildOperationFiringBuildWorkerExecutor$ExecuteTasks.call(BuildOperationFiringBuildWorkerExecutor.java:54)
        at org.gradle.execution.BuildOperationFiringBuildWorkerExecutor$ExecuteTasks.call(BuildOperationFiringBuildWorkerExecutor.java:43)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66)
        at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59)
        at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73)
        at org.gradle.execution.BuildOperationFiringBuildWorkerExecutor.execute(BuildOperationFiringBuildWorkerExecutor.java:40)
        at org.gradle.internal.build.DefaultBuildLifecycleController.lambda$executeTasks$7(DefaultBuildLifecycleController.java:172)
        at org.gradle.internal.model.StateTransitionController.doTransition(StateTransitionController.java:258)
        at org.gradle.internal.model.StateTransitionController.lambda$tryTransition$8(StateTransitionController.java:185)
        at org.gradle.internal.work.DefaultSynchronizer.withLock(DefaultSynchronizer.java:44)
        at org.gradle.internal.model.StateTransitionController.tryTransition(StateTransitionController.java:185)
        at org.gradle.internal.build.DefaultBuildLifecycleController.executeTasks(DefaultBuildLifecycleController.java:172)
        at org.gradle.internal.build.DefaultBuildWorkGraphController$DefaultBuildWorkGraph.runWork(DefaultBuildWorkGraphController.java:209)
        at org.gradle.internal.work.DefaultWorkerLeaseService.withLocks(DefaultWorkerLeaseService.java:249)
        at org.gradle.internal.work.DefaultWorkerLeaseService.runAsWorkerThread(DefaultWorkerLeaseService.java:109)
        at org.gradle.composite.internal.DefaultBuildController.doRun(DefaultBuildController.java:172)
        at org.gradle.composite.internal.DefaultBuildController.access$000(DefaultBuildController.java:47)
        at org.gradle.composite.internal.DefaultBuildController$BuildOpRunnable.run(DefaultBuildController.java:191)
        at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64)
        at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49)
Caused by: java.lang.StackOverflowError
        at com.github.leandroborgesferreira.dagcommand.logic.GraphKt.traverseGraph(Graph.kt:20)
        at com.github.leandroborgesferreira.dagcommand.logic.GraphKt.traverseGraph(Graph.kt:20)
        at com.github.leandroborgesferreira.dagcommand.logic.GraphKt.traverseGraph(Graph.kt:20)
        at com.github.leandroborgesferreira.dagcommand.logic.GraphKt.traverseGraph(Graph.kt:20)
        at com.github.leandroborgesferreira.dagcommand.logic.GraphKt.traverseGraph(Graph.kt:20)

* Get more help at https://help.gradle.org

Deprecated Gradle features were used in this build, making it incompatible with Gradle 9.0.

You can use '--warning-mode all' to show the individual deprecation warnings and determine if they come from your own scripts or plugins.

See https://docs.gradle.org/8.0/userguide/command_line_interface.html#sec:command_line_warnings

BUILD FAILED in 3s
leandroBorgesFerreira commented 1 year ago

Okay. I'll reproduce on my side and create a fix.

Meanwhile, are you using the plugin for running your UI tests in parallel, right? I believe that setting printModulesInfo = false should make it work for that purpose. Use it as a workaround while I fix the information print.

leandroBorgesFerreira commented 1 year ago

Actually. It won't fix it, because the Stackoverflow is happening when the affected modules are being generated. By reading your logs, I see that the adjacency list of your modules is being printed in your /Users/sagarkhurana/androidsecond/android/build path. Could you share the adjacency list with me? It makes it much easier to see where the cycle is happening.

hellosagar commented 1 year ago

Here's the output of adjacency list

Edit by Leandro [Erased for privacy reasons]

VolodymyrMachekhin commented 1 year ago

So the cycle happens because module benchmark in the build.gradle sets targetProjectPath = ":getyourguide" (as the application module). Andgetyourguide/build.gradle.ktshas the importbaselineProfile(project(Project.BENCHMARK)). When we commentbaselineProfile(project(Project.BENCHMARK))` then the cycle disappears.

VolodymyrMachekhin commented 1 year ago

@leandroBorgesFerreira I would recommend to ignore benchmark module in private fun traverseGraph(adjacencyList: AdjacencyList, module: String, resultSet: MutableSet<String>) Would you accept such a pull request? Maybe even specify the ignoreModules = [...] @hellosagar do you know if benchmark has some unique configurations that could be checked or it has just the name? Also, is it possible to rename it?

leandroBorgesFerreira commented 1 year ago

Thanks for sharing @hellosagar.

I erased your comment to avoid exposing your dependency graph. I'll later change the name of the modules (rename then to A, B, C, D...) and include them in the unit tests for this library if you authorize me.

leandroBorgesFerreira commented 1 year ago

@VolodymyrMachekhin Sure a PR is more than welcome and a field to exclude some modules makes perfect sense to the library!

If you open a PR to it, I can merge it with no problems. I just created the version 1.8.0 (I lost the 1.7.0 while changing to gradle plugin portal instead of maven central), which already detects cycles in graphs, which should help the development of your PR.

leandroBorgesFerreira commented 1 year ago

@VolodymyrMachekhin Just for documentation: You're 100% right. The benchmark module points to the main app module, which points back to it. That needs to be fixed.

hellosagar commented 1 year ago

@leandroBorgesFerreira I would like to pick up this task, to exclude some modules using the configuration. Might need some help on my way 😄

leandroBorgesFerreira commented 1 year ago

Sure @hellosagar. I'm glad that you would like to contribute.

First, this is how you're going to test to progress to make sure you fixed it for your project:

You can also add new configuration and log it, to make sure you are passing the configuration in the right way.

leandroBorgesFerreira commented 1 year ago

This is where you should add your configuration new field in the configuration (excludeModules or something like it) CommandExtension and also here Config.

Then you do some new filtering in this file: ProjectParse.kt. Or any other file that you see fit. Feel free to do the PR the way you believe it is best.

Happy coding! =D

leandroBorgesFerreira commented 6 months ago

Hello hello.

Prior the version 1.10.0, you can filter modules out. Just use the filterModules configuration and the modules will be removed from the dag when using the plugin.

hellosagar commented 6 months ago

@leandroBorgesFerreira Failing with the Stackoverflow exception, after using the filterModules as well

Dag config

project.extensions.getByType(io.github.leandroborgesferreira.dagcommand.extension.CommandExtension::class.java)
  .apply {
    defaultBranch = "origin/develop"
    outputType = "json"
    printModulesInfo = true
    filterModules = "[\"filter:benchmark!\"]"
  }
Removed
leandroBorgesFerreira commented 6 months ago

@hellosagar.

Your configuration seams to be incorrect. Use this: 

project.extensions.getByType(io.github.leandroborgesferreira.dagcommand.extension.CommandExtension::class.java)
  .apply {
    defaultBranch = "origin/develop"
    outputType = "json"
    printModulesInfo = true
    filterModules = "[\"benchmark\"]"
  }

Basically you put the name of the module instead of filter:[module]!

hellosagar commented 6 months ago

@leandroBorgesFerreira After using the correct filter, Still facing

project.extensions.getByType(io.github.leandroborgesferreira.dagcommand.extension.CommandExtension::class.java)
  .apply {
    defaultBranch = "origin/develop"
    outputType = "json"
    printModulesInfo = true
    filterModules = "[\"benchmark\"]"
  }

Output:

Removed
leandroBorgesFerreira commented 6 months ago

Looks like the stackoverflow is happening before the filter 🤦🏽. Just a sec, I'll move the filter a bit earlier in the code.

leandroBorgesFerreira commented 6 months ago

@hellosagar I just publish the version 1.10.1 of the plugin. It should not enter the cycle now.

hellosagar commented 6 months ago

@leandroBorgesFerreira , it worked! Thank you very much. I really appreciate that 🤝

leandroBorgesFerreira commented 6 months ago

Uhuuuu!!

Good to know that it worked! Thanks for posting the issue, it made much easier to find the bug.