gradle clean doesn't leave all projects clean in a pride

[jzwolak]

In my pride I ran gradle -m clean and the output is:

:processdb:clean SKIPPED :processdb_rest_generator:clean SKIPPED :processdb:tspstuffJar SKIPPED :processdb_rest_generator:generator:compileJava SKIPPED :processdb_rest_generator:generator:processResources SKIPPED :processdb_rest_generator:generator:classes SKIPPED :processdb_rest_generator:reversemap:generateHibernateReverseEngineerXml SKIPPED :processdb_rest_generator:reversemap:reverseMap SKIPPED :processdb_rest_generator:reversemap:compileJava SKIPPED :processdb_rest_generator:reversemap:processResources SKIPPED :processdb_rest_generator:reversemap:classes SKIPPED :processdb_rest_generator:common:generateEntityContainers SKIPPED :processdb_rest_generator:reversemap:jar SKIPPED :processdb_rest_generator:common:compileJava SKIPPED :processdb_rest_generator:common:processResources SKIPPED :processdb_rest_generator:common:classes SKIPPED :processdb_rest_generator:common:jar SKIPPED :processdb_rest_service:grails-clean SKIPPED :processdb_rest_service:clean SKIPPED :processdb_rest_generator:client:clean SKIPPED :processdb_rest_generator:common:clean SKIPPED :processdb_rest_generator:generator:clean SKIPPED :processdb_rest_generator:reversemap:clean SKIPPED :processdb_rest_generator:service:clean SKIPPED

Of note are the 1st and 3rd lines. The processdb project is cleaned and on the 3rd line the tspstuffJar task is run which creates an artifact in the project... that is, the project is no longer clean.

The tspstuffJar task is executed because the processdb rest generator project depends on processdb's tspstuffJar (via the configuration option in the dynamic dependencies for compile).

I did a test and removed the configuration option. That changed the dynamic dependencies of processdb rest generator project from:

    dynamicDependencies {
        runtime group: "com.integrativebioinformatics", name: "processdb",
            version: "1.0", configuration: "tspstuff"
    }

to

    dynamicDependencies {
        runtime group: "com.integrativebioinformatics", name: "processdb",
            version: "1.0"
    }

And in fact the new output runs the compileJava task of the processdb project after running the clean task... here are the first three lines:

:processdb:clean SKIPPED :processdb_rest_generator:clean SKIPPED :processdb:compileJava SKIPPED

(note: I used the '-m' flag again to skip the execution of the tasks and just print them)

It seems that the pride should execute the clean task on the leaves of the dependency tree last in order to leave all projects in the "clean" state. Doing so makes the clean task special and different from the other tasks.

Perhaps Pride can have the concept of a task that runs in "reverse" where the dependencies are evaluated normally for execution of a specific task but the order pride calls the task on the modules is ensured to be the reverse of the dependency tree between the modules (from root to leaves instead of leaves to root). In this case the user could specify which tasks to run in reverse and the clean task can default to reverse.

I can think of two other ways to do this and don't know the best approach. One other way is to not evaluate dependencies for tasks like the clean task. Another is to detect the projects that are compiled because of a dynamic dependency and rerun the clean task for those projects after the first run... this approach screams problems to me, though.

There's another problem besides simply not having a clean project after running clean. If a dependency won't compile then the clean will fail on the dependent project. Logically, this should not happen. However, it is possible that the build script really needs something from a dependency in order to do the clean... that is, if the author of the build script wrote it in. This has to do with the way Gradle evaluates dependencies.

I wonder if Gradle's support for multi-project builds solves this issue in a way that can be borrowed for Pride. In Gradle a new task doesn't depend on any projects unless specified in the "dependsOn" argument. Doing this in Pride would break the ability to have a single build script that works in and out of a pride. Perhaps another extension like dynamicDependencies could be used to resolve this issue.

[jzwolak]

Or maybe the generation of the dependencies artifact can be delayed until it is used somehow. Since the clean task won't use it... it won't be generated.

This approach may be impossible... but it depends on how Gradle includes the artifact in the Java classpath when executing tasks. Gradle must have some smart mechanisms for this since there are different configurations with different classpaths. Maybe Pride can detect when a configuration is actually used and trigger the execution of the tasks to build the artifact then. Or maybe Pride can detect when a configuration is used by a task in the current task graph.

[lptr]

Is this something related to Pride at all? What happens if you try to do the same without Pride?

[jzwolak]

Yes it is related to Pride.

In Gradle multi-project builds this doesn't happen. And since Gradle doesn't support dynamic dependencies, like Pride does, Gradle won't trigger a compile of the dependencies to run a task.

The problem is that a task might need the dependencies to run. Therefore, Pride correctly has Gradle compile the dependencies. But, during a clean, the dependencies should be left in the after-clean state not the after-compile state. At least that's what I expected. Currently, this would only happen by chance based on the order of the clean tasks run for each project.

[lptr]

Can you create a small test case where I can reproduce the problem?

[jzwolak]

I tried to create a small test case... but in my small test case I cannot even add the repositories to the pride. Maybe if I have some more time I'll figure it out later.

[lptr]

Closing this as the problem has not been reproduced by any of a dozen developers in the last months. Please reopen if you bump into this again.