search

aaschmid / gradle-cpd-plugin

Gradle plugin to find duplicate code using PMDs copy/paste detection (= CPD).
Apache License 2.0
95 stars 12 forks source link

Use Gradle's Worker API #32

Closed larsgrefer closed 5 years ago

larsgrefer commented 5 years ago

This removes the need for the classloading hack mentioned here:https://github.com/aaschmid/gradle-cpd-plugin/blob/ed44ed7727edca3e404738ebf3e84658156d5e57/src/main/groovy/de/aaschmid/gradle/plugins/cpd/Cpd.groovy#L183

As addition, the task can now run parallel to other tasks, and multiple reports can be generated at once.

See also: https://guides.gradle.org/using-the-worker-api/

aaschmid commented 5 years ago

@larsgrefer very cool :-)

Can you agree to the terms of the Gradle CPD plugin Contributor License Agreement, also for this PR?

I will review it asap.

larsgrefer commented 5 years ago

@aaschmid Some of the Tests in de.aaschmid.gradle.plugins.cpd.test.CpdAcceptanceTest aren't working, because it's not possible anymore to just call Task.execute(). These tests should be moved to de.aaschmid.gradle.plugins.cpd.test.CpdIntegrationTest and rewritten to use the Gradle TestKit (org.gradle.testkit.runner.GradleRunner).

aaschmid commented 5 years ago

@larsgrefer hm ... yes I am aware of that change - seems that Gradle does not like unit tests ... However, another way is to get the actions of the task and execute these actions. Do you want to try?

aaschmid commented 5 years ago

@larsgrefer Thanks for the work. It looks very well, already. Please don't be angry / afraid about all the review comments, I am happy to discuss them or even implement them myself if you prefer :-)

larsgrefer commented 5 years ago

@larsgrefer hm ... yes I am aware of that change - seems that Gradle does not like unit tests ... However, another way is to get the actions of the task and execute these actions. Do you want to try?

Some of the tests (mostly the ones testing successful cases) could be refactored to directly use the new CpdAction, but some tests which actually test the task itself, should be refactored to use Gradle's TestKit.

aaschmid commented 5 years ago

After merging the code, tests fail with an exception like the following. I will investigate further ...

de.aaschmid.gradle.plugins.cpd.test.CpdAcceptanceTest > applying 'Cpd' task to only parent project if only sub project has 'groovy' plugin FAILED
    org.gradle.api.tasks.TaskExecutionException: Execution failed for task ':cpdCheck'.
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:110)
        at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:77)
        at org.gradle.api.internal.tasks.execution.OutputDirectoryCreatingTaskExecuter.execute(OutputDirectoryCreatingTaskExecuter.java:51)
        at org.gradle.api.internal.tasks.execution.SkipUpToDateTaskExecuter.execute(SkipUpToDateTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.ResolveTaskOutputCachingStateExecuter.execute(ResolveTaskOutputCachingStateExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ValidatingTaskExecuter.execute(ValidatingTaskExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.SkipEmptySourceFilesTaskExecuter.execute(SkipEmptySourceFilesTaskExecuter.java:101)
        at org.gradle.api.internal.tasks.execution.FinalizeInputFilePropertiesTaskExecuter.execute(FinalizeInputFilePropertiesTaskExecuter.java:44)
        at org.gradle.api.internal.tasks.execution.CleanupStaleOutputsExecuter.execute(CleanupStaleOutputsExecuter.java:91)
        at org.gradle.api.internal.tasks.execution.ResolveTaskArtifactStateTaskExecuter.execute(ResolveTaskArtifactStateTaskExecuter.java:62)
        at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:59)
        at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:54)
        at org.gradle.api.internal.tasks.execution.ExecuteAtMostOnceTaskExecuter.execute(ExecuteAtMostOnceTaskExecuter.java:43)
        at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:34)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.run(EventFiringTaskExecuter.java:51)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:301)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:293)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:175)
        at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
        at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
        at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:46)
        at org.gradle.api.internal.AbstractTask.execute(AbstractTask.java:402)
        at de.aaschmid.gradle.plugins.cpd.test.CpdAcceptanceTest.applying 'Cpd' task to only parent project if only sub project has 'groovy' plugin(CpdAcceptanceTest.groovy:206)
        Caused by:
        org.gradle.workers.internal.DefaultWorkerExecutor$WorkExecutionException: A failure occurred while executing de.aaschmid.gradle.plugins.cpd.internal.CpdAction
            at org.gradle.workers.internal.DefaultWorkerExecutor.submit(DefaultWorkerExecutor.java:87)
            at de.aaschmid.gradle.plugins.cpd.Cpd.run(Cpd.groovy:203)
            at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:73)
            at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:46)
            at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:39)
            at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:26)
            at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:801)
            at org.gradle.api.internal.AbstractTask$TaskActionWrapper.execute(AbstractTask.java:768)
            at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter$1.run(ExecuteActionsTaskExecuter.java:131)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:301)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor$RunnableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:293)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:175)
            at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:91)
            at org.gradle.internal.operations.DelegatingBuildOperationExecutor.run(DelegatingBuildOperationExecutor.java:31)
            at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeAction(ExecuteActionsTaskExecuter.java:120)
            at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeActions(ExecuteActionsTaskExecuter.java:99)
            ... 22 more
            Caused by:
            org.gradle.workers.internal.SerializingActionExecutionSpec$ParameterSerializationException: Could not serialize parameters
                at org.gradle.workers.internal.SerializingActionExecutionSpec.serialize(SerializingActionExecutionSpec.java:71)
                at org.gradle.workers.internal.SerializingActionExecutionSpec.<init>(SerializingActionExecutionSpec.java:42)
                at org.gradle.workers.internal.DefaultWorkerExecutor.submit(DefaultWorkerExecutor.java:85)
                ... 37 more
                Caused by:
                java.io.NotSerializableException: org.gradle.api.internal.provider.DefaultPropertyState
                    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1184)
                    at java.io.ObjectOutputStream.defaultWriteFields(ObjectOutputStream.java:1548)
                    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1509)
                    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
                    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
                    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
                    at java.util.ArrayList.writeObject(ArrayList.java:766)
                    at java.io.ObjectStreamClass.invokeWriteObject(ObjectStreamClass.java:1140)
                    at java.io.ObjectOutputStream.writeSerialData(ObjectOutputStream.java:1496)
                    at java.io.ObjectOutputStream.writeOrdinaryObject(ObjectOutputStream.java:1432)
                    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1178)
                    at java.io.ObjectOutputStream.writeArray(ObjectOutputStream.java:1378)
                    at java.io.ObjectOutputStream.writeObject0(ObjectOutputStream.java:1174)
                    at java.io.ObjectOutputStream.writeObject(ObjectOutputStream.java:348)
                    at org.gradle.workers.internal.SerializingActionExecutionSpec.serialize(SerializingActionExecutionSpec.java:69)
                    ... 39 more
aaschmid commented 5 years ago

Problem is that properties of Cpd*FileReportImpl - which are used by Gradle - are not serializeable in the end. Therefore I build a mapping to simple DTOs for them. Still on it ...

aaschmid commented 5 years ago

Problem is that properties of Cpd*FileReportImpl - which are used by Gradle - are not serializeable in the end. Therefore I build a mapping to simple DTOs for them.

Now a java.util.ServiceConfigurationError appears in most unit tests :-( Even though integration test work, packaging the plugin and use it another projects fails it...

java.util.ServiceConfigurationError: net.sourceforge.pmd.cpd.Language: Provider net.sourceforge.pmd.cpd.CPPLanguage not a subtype
    at java.util.ServiceLoader.fail(ServiceLoader.java:239)
    at java.util.ServiceLoader.access$300(ServiceLoader.java:185)
    at java.util.ServiceLoader$LazyIterator.nextService(ServiceLoader.java:376)
    at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:404)
    at java.util.ServiceLoader$1.next(ServiceLoader.java:480)
    at net.sourceforge.pmd.cpd.LanguageFactory.<init>(LanguageFactory.java:27)
    at net.sourceforge.pmd.cpd.LanguageFactory.<clinit>(LanguageFactory.java:16)
    at de.aaschmid.gradle.plugins.cpd.internal.CpdAction.run(CpdAction.java:78)
    at org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:39)
    at org.gradle.workers.internal.DefaultWorkerServer.execute(DefaultWorkerServer.java:25)
    at org.gradle.workers.internal.IsolatedClassloaderWorkerFactory$WorkerCallable.call(IsolatedClassloaderWorkerFactory.java:177)
    at org.gradle.workers.internal.IsolatedClassloaderWorkerFactory.executeInWorkerClassLoader(IsolatedClassloaderWorkerFactory.java:105)
    at org.gradle.workers.internal.IsolatedClassloaderWorkerFactory.access$100(IsolatedClassloaderWorkerFactory.java:54)
    at org.gradle.workers.internal.IsolatedClassloaderWorkerFactory$1$1.call(IsolatedClassloaderWorkerFactory.java:78)
    at org.gradle.workers.internal.IsolatedClassloaderWorkerFactory$1$1.call(IsolatedClassloaderWorkerFactory.java:75)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:315)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor$CallableBuildOperationWorker.execute(DefaultBuildOperationExecutor.java:305)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.execute(DefaultBuildOperationExecutor.java:175)
    at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:101)
    at org.gradle.internal.operations.DelegatingBuildOperationExecutor.call(DelegatingBuildOperationExecutor.java:36)
    at org.gradle.workers.internal.IsolatedClassloaderWorkerFactory$1.execute(IsolatedClassloaderWorkerFactory.java:75)
    at org.gradle.workers.internal.DefaultWorkerExecutor$1.call(DefaultWorkerExecutor.java:102)
    at org.gradle.workers.internal.DefaultWorkerExecutor$1.call(DefaultWorkerExecutor.java:96)
    at org.gradle.internal.work.AbstractConditionalExecution$1.run(AbstractConditionalExecution.java:38)
    at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.runExecution(DefaultConditionalExecutionQueue.java:212)
    at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.runBatch(DefaultConditionalExecutionQueue.java:161)
    at org.gradle.internal.work.DefaultConditionalExecutionQueue$ExecutionRunner.run(DefaultConditionalExecutionQueue.java:130)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:63)
    at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:46)
    at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at org.gradle.internal.concurrent.ThreadFactoryImpl$ManagedThreadRunnable.run(ThreadFactoryImpl.java:55)
    at java.lang.Thread.run(Thread.java:748)