Open jmini opened 10 months ago
/cc @evanchooly (kotlin), @geoand (kotlin), @glefloch, @maxandersen (jbang), @quarkusio/devtools (jbang)
Some more inputs to narrow the problem down:
if we look at the *.class
in the core-lib
project:
core-lib/build/classes/java/main/org/acme/core/TimestampMessage.class
core-lib/build/classes/java/main/org/acme/core/CoreProcessor.class
core-lib/build/classes/java/main/org/acme/core/CoreMessage.class
core-lib/build/classes/kotlin/main/org/acme/core/kt/ConfigService.class
We see that there are 2 folders:
build/classes/java
build/classes/kotlin
I have the feeling that Quarkus is only looking at build/classes/java
I didn't look yet at how this is implemented, but when you ask Gradle (inside a gradle task/plugin) about the output of the main sourceSet:
JavaPluginExtension extension = project.getExtensions().findByType(JavaPluginExtension.class);
SourceSet mainSourceSet = extension.getSourceSets().getByName("main");
for (File f : mainSourceSet.getOutput().getFiles()) {
System.out.println(f);
}
you see the different folders at output of the main source-set:
See https://github.com/jmini/quarkus_issue35577/commit/d9f765e19728e5f160a0b1cfe149526b79c1fec0 that adds a printInfo
task to the gradle modules.
An other experiment to work-around this issue:
When I add a task that copy the content of build/classes/kotlin
into build/classes/java
with an additional task: copyReportKtClasses
(see https://github.com/jmini/quarkus_issue35577/commit/02c7b6b67aaa867086c177927ed071a62855bef0)
Then it seems that quarkus dev-mode is working correctly:
./gradlew app1:quarkusDev
I also had to:
quarkusDependenciesBuild
depends on my new taskjar
task since now when the jar is built, all the kotlin classes are duplicatesAnd of course for gradle this strategy is really wrong, since it break all the input/output concept. And the compile task will not be UP_TO_DATE since my copy task is changing the output of the compile-java task.
@jmini maybe you can have a look at what the Gradle plugin is doing and provide a fix? That would be helpful.
I did not dig into that part of the code yet. :-)
A very naive search pointed to this test: https://github.com/quarkusio/quarkus/blob/a2517e87b9070c282bb397630b927cdd4128f4bd/devtools/gradle/gradle-application-plugin/src/test/java/io/quarkus/gradle/QuarkusPluginTest.java#L89-L104
Which make me think my use case should be supported... But probably this is a complete different step and not related to my issue.
Do you have some implementation details/documentation about how the gradle plugin is working (which steps create which output and what is the difference between devMode and production build)?
I would like to narrow down the part that is not working.
Maybe the issue is in the GradleApplicationModelBuilder, but I am not sure how to continue my investigations.
Should I verify if the application model is as expected? Or different between production mode and dev mode?
Should I verify if the classpath of the started application is correct? (for instance that both core-lib/build/classes/java/main
and core-lib/build/classes/kotlin/main
are part of the classpath).
I have tried to do a jinfo <pid>
on the process started by gradle to serve 8080
.
The classpath seems only to be app1/build/app1-dev.jar
Can the application model app1/build/tmp/quarkusDev/quarkus-app-model.dat
be investigated (pretty print to display it as text)?
Can the application model be investigated?
I started a jbang script to do this: QuarkusAppMain.java
It could probably be improved to display the values in a nice way, but I think this is enough to already see that gradle did not create the app model correctly.
When I deserialized the quarkus-app-model.dat
into appModel
and check the content of appModel.getDependencies()
, the first item in that collection is:
org.acme:core-lib::jar:1.0.0-SNAPSHOT[paths: /<quarkus_issue35577 path>/core-lib/build/classes/java/main;/<quarkus_issue35577 path>/core-lib/build/resources/main;] io.quarkus.bootstrap.workspace.DefaultWorkspaceModule$Builder@53bd815b
I think the build/classes/kotlin/main
path should be part of the PathCollection
returned by getResolvedPaths()
of the ResolvedDependency
instance corresponding to the core-lib
dependency.
@jmini yes you are right, the path build/classes/kotlin/main
is missing. I'm looking on how we could detect the kotlin folder to add it.
The workaround I proposed couple of days ago is not really working in all cases.
I did not find the correct moment where copyReportKtClasses
should run. I am now at:
//Workaround for https://github.com/quarkusio/quarkus/issues/35577:
def copyReportKtClassesTask = tasks.register('copyReportKtClasses', Copy) {
dependsOn tasks.named('compileKotlin')
from layout.buildDirectory.file('classes/kotlin')
into layout.buildDirectory.file('classes/java/')
}
jar {
duplicatesStrategy(DuplicatesStrategy.EXCLUDE)
}
tasks.named("compileJava") {
dependsOn copyReportKtClassesTask
}
//End of Workaround for https://github.com/quarkusio/quarkus/issues/35577
This works in the reproducer project: https://github.com/jmini/quarkus_issue35577
But in our real application I am still having hard time to find why it doesn't work. The UnsatisfiedResolutionException
is still present.
I did some debugging into the issue.
Breakpoint is at maybeConfigureKotlinJvmCompile(..)
:
When we are building the model for :app1
when we are collecting the dependencies.
So project
instance corresponds to project ':core-lib'
```
Thread [Execution worker Thread 4] (Suspended (entry into method maybeConfigureKotlinJvmCompile in GradleApplicationModelBuilder))
GradleApplicationModelBuilder.maybeConfigureKotlinJvmCompile(Project, FileCollection, List
From a gradle task point of view we are at starting the quarkusGenerateCodeDev
task in the :app1
module:
> Task :app1:processResources
> Task :core-lib:processResources
> Task :core-lib:quarkusGenerateCode
> Task :core-lib:quarkusGenerateCodeDev
> Task :app1:processTestResources NO-SOURCE
> Task :app1:compileQuarkus-test-generated-sourcesKotlin NO-SOURCE
> Task :app1:compileQuarkusTestGeneratedSourcesJava NO-SOURCE
> Task :core-lib:compileKotlin
> Task :core-lib:compileJava
> Task :core-lib:classes
> Task :core-lib:jandex
> Task :core-lib:jar
> Task :app1:quarkusGenerateCode
> Task :app1:quarkusGenerateCodeDev
First finding:
The way maybeConfigureKotlinJvmCompile(..)
requires the module :app1
to have the kotlin plugin installed.
Otherwise the method is skipped because of the try-catch:
``` java.lang.ClassNotFoundException: org.jetbrains.kotlin.gradle.tasks.KotlinJvmCompile at java.base/java.net.URLClassLoader.findClass(URLClassLoader.java:445) at org.gradle.internal.classloader.VisitableURLClassLoader$InstrumentingVisitableURLClassLoader.findClass(VisitableURLClassLoader.java:186) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:587) at java.base/java.lang.ClassLoader.loadClass(ClassLoader.java:520) at java.base/java.lang.Class.forName0(Native Method) at java.base/java.lang.Class.forName(Class.java:375) at io.quarkus.gradle.tooling.GradleApplicationModelBuilder.maybeConfigureKotlinJvmCompile(GradleApplicationModelBuilder.java:527) at io.quarkus.gradle.tooling.GradleApplicationModelBuilder.initProjectModule(GradleApplicationModelBuilder.java:487) at io.quarkus.gradle.tooling.GradleApplicationModelBuilder.initProjectModuleAndBuildPaths(GradleApplicationModelBuilder.java:403) at io.quarkus.gradle.tooling.GradleApplicationModelBuilder.collectDependencies(GradleApplicationModelBuilder.java:337) at io.quarkus.gradle.tooling.GradleApplicationModelBuilder.lambda$collectDependencies$4(GradleApplicationModelBuilder.java:243) at java.base/java.lang.Iterable.forEach(Iterable.java:75) at io.quarkus.gradle.tooling.GradleApplicationModelBuilder.collectDependencies(GradleApplicationModelBuilder.java:242) at io.quarkus.gradle.tooling.GradleApplicationModelBuilder.buildAll(GradleApplicationModelBuilder.java:123) at io.quarkus.gradle.tooling.ToolingUtils.create(ToolingUtils.java:74) at io.quarkus.gradle.tooling.ToolingUtils.create(ToolingUtils.java:70) at io.quarkus.gradle.extension.QuarkusPluginExtension.getApplicationModel(QuarkusPluginExtension.java:181) at io.quarkus.gradle.tasks.QuarkusDev.newLauncher(QuarkusDev.java:447) at io.quarkus.gradle.tasks.QuarkusDev.startDev(QuarkusDev.java:342) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at org.gradle.internal.reflect.JavaMethod.invoke(JavaMethod.java:125) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.doExecute(StandardTaskAction.java:58) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:51) at org.gradle.api.internal.project.taskfactory.StandardTaskAction.execute(StandardTaskAction.java:29) at org.gradle.api.internal.tasks.execution.TaskExecution$3.run(TaskExecution.java:242) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:29) at org.gradle.internal.operations.DefaultBuildOperationRunner$1.execute(DefaultBuildOperationRunner.java:26) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.run(DefaultBuildOperationRunner.java:47) at org.gradle.internal.operations.DefaultBuildOperationExecutor.run(DefaultBuildOperationExecutor.java:68) at org.gradle.api.internal.tasks.execution.TaskExecution.executeAction(TaskExecution.java:227) at org.gradle.api.internal.tasks.execution.TaskExecution.executeActions(TaskExecution.java:210) at org.gradle.api.internal.tasks.execution.TaskExecution.executeWithPreviousOutputFiles(TaskExecution.java:193) at org.gradle.api.internal.tasks.execution.TaskExecution.execute(TaskExecution.java:166) at org.gradle.internal.execution.steps.ExecuteStep.executeInternal(ExecuteStep.java:105) at org.gradle.internal.execution.steps.ExecuteStep.access$000(ExecuteStep.java:44) at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:59) at org.gradle.internal.execution.steps.ExecuteStep$1.call(ExecuteStep.java:56) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53) at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73) at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:56) at org.gradle.internal.execution.steps.ExecuteStep.execute(ExecuteStep.java:44) at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:67) at org.gradle.internal.execution.steps.RemovePreviousOutputsStep.execute(RemovePreviousOutputsStep.java:37) at org.gradle.internal.execution.steps.CancelExecutionStep.execute(CancelExecutionStep.java:41) at org.gradle.internal.execution.steps.TimeoutStep.executeWithoutTimeout(TimeoutStep.java:74) at org.gradle.internal.execution.steps.TimeoutStep.execute(TimeoutStep.java:55) at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:50) at org.gradle.internal.execution.steps.CreateOutputsStep.execute(CreateOutputsStep.java:28) at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.executeDelegateBroadcastingChanges(CaptureStateAfterExecutionStep.java:100) at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:72) at org.gradle.internal.execution.steps.CaptureStateAfterExecutionStep.execute(CaptureStateAfterExecutionStep.java:50) at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:40) at org.gradle.internal.execution.steps.ResolveInputChangesStep.execute(ResolveInputChangesStep.java:29) at org.gradle.internal.execution.steps.BuildCacheStep.executeWithoutCache(BuildCacheStep.java:166) at org.gradle.internal.execution.steps.BuildCacheStep.lambda$execute$1(BuildCacheStep.java:70) at org.gradle.internal.Either$Right.fold(Either.java:175) at org.gradle.internal.execution.caching.CachingState.fold(CachingState.java:59) at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:68) at org.gradle.internal.execution.steps.BuildCacheStep.execute(BuildCacheStep.java:46) at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:36) at org.gradle.internal.execution.steps.StoreExecutionStateStep.execute(StoreExecutionStateStep.java:25) at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:36) at org.gradle.internal.execution.steps.RecordOutputsStep.execute(RecordOutputsStep.java:22) at org.gradle.internal.execution.steps.SkipUpToDateStep.executeBecause(SkipUpToDateStep.java:91) at org.gradle.internal.execution.steps.SkipUpToDateStep.lambda$execute$2(SkipUpToDateStep.java:55) at java.base/java.util.Optional.orElseGet(Optional.java:364) at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:55) at org.gradle.internal.execution.steps.SkipUpToDateStep.execute(SkipUpToDateStep.java:37) at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:65) at org.gradle.internal.execution.steps.ResolveChangesStep.execute(ResolveChangesStep.java:36) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:37) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsFinishedStep.execute(MarkSnapshottingInputsFinishedStep.java:27) at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:76) at org.gradle.internal.execution.steps.ResolveCachingStateStep.execute(ResolveCachingStateStep.java:37) at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:94) at org.gradle.internal.execution.steps.ValidateStep.execute(ValidateStep.java:49) at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:71) at org.gradle.internal.execution.steps.CaptureStateBeforeExecutionStep.execute(CaptureStateBeforeExecutionStep.java:45) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.executeWithNonEmptySources(SkipEmptyWorkStep.java:177) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:81) at org.gradle.internal.execution.steps.SkipEmptyWorkStep.execute(SkipEmptyWorkStep.java:53) at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:32) at org.gradle.internal.execution.steps.RemoveUntrackedExecutionStateStep.execute(RemoveUntrackedExecutionStateStep.java:21) at org.gradle.internal.execution.steps.legacy.MarkSnapshottingInputsStartedStep.execute(MarkSnapshottingInputsStartedStep.java:38) at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:36) at org.gradle.internal.execution.steps.LoadPreviousExecutionStateStep.execute(LoadPreviousExecutionStateStep.java:23) at org.gradle.internal.execution.steps.CleanupStaleOutputsStep.execute(CleanupStaleOutputsStep.java:75) at org.gradle.internal.execution.steps.CleanupStaleOutputsStep.execute(CleanupStaleOutputsStep.java:41) at org.gradle.internal.execution.steps.AssignWorkspaceStep.lambda$execute$0(AssignWorkspaceStep.java:32) at org.gradle.api.internal.tasks.execution.TaskExecution$4.withWorkspace(TaskExecution.java:287) at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:30) at org.gradle.internal.execution.steps.AssignWorkspaceStep.execute(AssignWorkspaceStep.java:21) at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:37) at org.gradle.internal.execution.steps.IdentityCacheStep.execute(IdentityCacheStep.java:27) at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:47) at org.gradle.internal.execution.steps.IdentifyStep.execute(IdentifyStep.java:34) at org.gradle.internal.execution.impl.DefaultExecutionEngine$1.execute(DefaultExecutionEngine.java:64) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.executeIfValid(ExecuteActionsTaskExecuter.java:146) at org.gradle.api.internal.tasks.execution.ExecuteActionsTaskExecuter.execute(ExecuteActionsTaskExecuter.java:135) at org.gradle.api.internal.tasks.execution.FinalizePropertiesTaskExecuter.execute(FinalizePropertiesTaskExecuter.java:46) at org.gradle.api.internal.tasks.execution.ResolveTaskExecutionModeExecuter.execute(ResolveTaskExecutionModeExecuter.java:51) at org.gradle.api.internal.tasks.execution.SkipTaskWithNoActionsExecuter.execute(SkipTaskWithNoActionsExecuter.java:57) at org.gradle.api.internal.tasks.execution.SkipOnlyIfTaskExecuter.execute(SkipOnlyIfTaskExecuter.java:74) at org.gradle.api.internal.tasks.execution.CatchExceptionTaskExecuter.execute(CatchExceptionTaskExecuter.java:36) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.executeTask(EventFiringTaskExecuter.java:77) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:55) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter$1.call(EventFiringTaskExecuter.java:52) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:204) at org.gradle.internal.operations.DefaultBuildOperationRunner$CallableBuildOperationWorker.execute(DefaultBuildOperationRunner.java:199) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:66) at org.gradle.internal.operations.DefaultBuildOperationRunner$2.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:157) at org.gradle.internal.operations.DefaultBuildOperationRunner.execute(DefaultBuildOperationRunner.java:59) at org.gradle.internal.operations.DefaultBuildOperationRunner.call(DefaultBuildOperationRunner.java:53) at org.gradle.internal.operations.DefaultBuildOperationExecutor.call(DefaultBuildOperationExecutor.java:73) at org.gradle.api.internal.tasks.execution.EventFiringTaskExecuter.execute(EventFiringTaskExecuter.java:52) at org.gradle.execution.plan.LocalTaskNodeExecutor.execute(LocalTaskNodeExecutor.java:42) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:337) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$InvokeNodeExecutorsAction.execute(DefaultTaskExecutionGraph.java:324) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:317) at org.gradle.execution.taskgraph.DefaultTaskExecutionGraph$BuildOperationAwareExecutionAction.execute(DefaultTaskExecutionGraph.java:303) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.execute(DefaultPlanExecutor.java:463) at org.gradle.execution.plan.DefaultPlanExecutor$ExecutorWorker.run(DefaultPlanExecutor.java:380) at org.gradle.internal.concurrent.ExecutorPolicy$CatchAndRecordFailures.onExecute(ExecutorPolicy.java:64) at org.gradle.internal.concurrent.ManagedExecutorImpl$1.run(ManagedExecutorImpl.java:49) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635) at java.base/java.lang.Thread.run(Thread.java:833) ```
In the reproducer only :core-lib
was a Kotlin project.
But the kotlin plugin can be added to the :app1
module as well.
Second finding:
Even after adding the kotlin plugin in :app1
, the quarkus gradle plugin is not working.
The reason is that when you ask for tasks withType(org.jetbrains.kotlin.gradle.tasks.KotlinJvmCompile.class)
of project :core-lib
inside the project :app1
the size of the returned collection is 0.
Verifier checkout the branch analysis
of the reproducer and compare the output of:
./gradlew printInfo1
This task is defined in the :core-lib
module
--> withType(org.jetbrains.kotlin.gradle.tasks.KotlinJvmCompile.class)
returns a task for the :core-lib
module.
and:
./gradlew printInfo2
This task is defined in the :app1
module, accessing the tasks of the other :core-lib
module.
--> withType(org.jetbrains.kotlin.gradle.tasks.KotlinJvmCompile.class)
finds nothing in the :core-lib
module accessed from :app1
@aloubyansky it is a place you seems to know (from issue https://github.com/quarkusio/quarkus/issues/20755) @glefloch what do you think?
At the end I am not sure why the io.quarkus.bootstrap.workspace.ArtifactSources
is defined like this with io.quarkus.bootstrap.workspace.SourceDir
expecting:
srcDir
destinationDir
Because the cardinality does not match, with the Gradle sourceSet model.
If you take a pure java project with 2 java source folders:
.
├── build.gradle
├── settings.gradle
├── src-1
│ └── Hello.java
└── src-2
└── World.java
with this build.gradle
:
plugins {
id 'java'
}
sourceSets {
main {
java {
srcDirs 'src-1', 'src-2'
}
}
}
The source set 'main'
defined in Gradle has one destination folder build/classes/java/main
but 2 source folders.
build/classes/
└── java
└── main
├── Hello.class
└── World.class
If we consider the source set 'main'
of :core-lib
in the reproducer, we could also consider that we have 2 destinations folders:
build/classes/java/main
build/classes/kotlin/main
Created by multiple source folders and multiple tasks (which is the information available in the ConfigurableFileCollection without too much hacks as it is now)
My current work-around is to take the allClassesDirs
and make sure there is at least one entry in the sourceDirs
of the quarkus model (where the classes-dir corresponds to the destinationDir
in the entry). The srcDir
of that entry does not really matter.
See my proposal: https://github.com/jmini/quarkus/commit/a426765314c79625e7fdc282b95f4be4b132cb61 and let me know if I should file that as a PR.
As discussed on Zulip:
I am wondering if the quarkus application model (io.quarkus.bootstrap.workspace.ArtifactSources
and io.quarkus.bootstrap.workspace.SourceDir
) is really compatible with the gradle model of SourceSet
The quarkus model:
ArtifactSources
you have to provide:
SourceDir
for sourcesSourceDir
for resourcesSourceDir
is expecting:
SourceDir#getDir()
SourceDir#getOutputDir()
io.quarkus.paths.PathTree
which gives more flexibility.The Gradle SourceSet model:
Example with a simple project with the kotlin extension (link to generate on code.quarkus.io). If I inspect the main SourceSet I have:
all sources srcDirs:
- /src/main/resources
- /src/main/java
- /build/classes/java/quarkus-generated-sources
- /build/classes/java/quarkus-generated-sources/grpc
- /build/classes/java/quarkus-generated-sources/avdl
- /build/classes/java/quarkus-generated-sources/avpr
- /build/classes/java/quarkus-generated-sources/avsc
- /src/main/kotlin
resources srcDirs:
- /src/main/resources
ouput ClassesDir:
- /build/classes/java/main
- /build/classes/kotlin/main
ouput ClassesDir builtBy:
- task ':compileKotlin'
- task ':compileJava'
And the are more complicated stuff that can be done in gradle (like more compile plugins or having a task that generate resources directly and register it to the main source set output with a builtBy
declaration)
My feeling is that because the Quarkus model is built like this, it is really difficult to map the Gradle model into the Quarkus one.
And this makes the implementation of GradleApplicationModelBuilder#initProjectModule(Project, Mutable, SourceSet, String)
really complex:
https://github.com/quarkusio/quarkus/blob/99ee1af61125b1b2575752ffc1e24ca3d12ff18d/devtools/gradle/gradle-model/src/main/java/io/quarkus/gradle/tooling/GradleApplicationModelBuilder.java#L472-L521
The complexity of this method (checking each task by type AbstractCompile
and KotlinJvmCompile
and checking the their input and outputs visitor methods) is because the current implementation tries to obtain from Gradle information that does not really exists (this is not how gradle works) to create the the Quarkus model.
Some notes on the Gradle API:
```gradle tasks.register('printInfo') { doLast { def mainSourceSet = sourceSets.main println(mainSourceSet) println("all sources srcDirs: ") mainSourceSet.getAllSource().getSrcDirs().each { println("- " + it.getAbsolutePath().substring(project.getRootProject().getProjectDir().getAbsolutePath().length() + 1)) } println("resources srcDirs: ") mainSourceSet.getResources().getSrcDirs().each { println("- " + it.getAbsolutePath().substring(project.getRootProject().getProjectDir().getAbsolutePath().length() + 1)) } println("ouput ClassesDir: ") mainSourceSet.getOutput().getClassesDirs().each { println("- " + it.getAbsolutePath().substring(project.getRootProject().getProjectDir().getAbsolutePath().length() + 1)) } println("ouput ClassesDir builtBy: ") (mainSourceSet.getOutput().getClassesDirs() as org.gradle.api.file.ConfigurableFileCollection).getBuiltBy().each { println("- " + it.get()) } } } ```
Are there any updates on this issue? This is quite a pain in a bigger project.
Personally I would love to contribute a patch, but I need some guidance:
I am not investing more time into this until I get some feedback from the committers responsible for the gradle plugin.
@aloubyansky @glefloch @snazy ^
Personally I would love to contribute a patch, but I need some guidance:
- to understand in which direction it should go (small hack or big change)
We might want to consider both, depending on the scope of the big change and how the small hack would look like. If a small hack looks like a quick reasonable change that would unblock people, let's consider that first and work on a bigger and proper change to fix it. I'll be interested in collaborating on that.
- where to put the integration test
Gradle ITs are in https://github.com/quarkusio/quarkus/tree/main/integration-tests/gradle. Given that the Quarkus project is Maven a project, Gradle ITs are actually launched from Maven. Meaning to run Gradle ITs from that module you could execute
mvn clean test
to run all the tests or
mvn clean test -Dtest=<test-class-name>
to run a specific test.
The test projects are located under main/resources
and copied to target/classes
and then also pre-processed as part of the test setup to set the appropriate Quarkus version matching the one in the main project.
I am not investing more time into this until I get some feedback from the committers responsible for the gradle plugin.
quarkusDev
is quite different. The current implementation basically defers all compilation to Quarkus code. This is (sadly) often wrong in (non trivial) Gradle projects that refer to non-standard project dependencies. By "non-standard" I mean dependencies to projects that are not Java and/or depend on a "non standard" configuration (like referencing a shadow-jar).
To fix that, there's a bunch of code that needs to be touched. IMO Gradle's continuous build functionality should be used to build the project including its dependencies. The Quarkus Dev code parts need to be changed to only reload changed parts but never actually build code.
I've started some work here, but haven't found time to continue on that (the code in that branch is hacky & messy). IIRC it could at least figure our the dependencies in a better way - but all the changes to the Quarkus Dev code are still missing. I don't mind if you want to pull some stuff from there.
Right, Gradle continuous build would behind quarkusDev be the ultimate longer term goal.
We might want to consider both, depending on the scope of the big change and how the small hack would look like.
@aloubyansky does https://github.com/jmini/quarkus/commit/a426765314c79625e7fdc282b95f4be4b132cb61 qualify as something that would be accepted?
If yes I could turn my example project quarkus_issue35577 into an integration test that would be failing before my patch and green after.
EDIT: the integration test part might be tricky since there is currently no tests testing :quarkusDev
@jmini yes, i think it would be acceptable. Thanks.
There is a way to reproduce it with ./gradlew quarkusIntTest
(after having added a test case to the reproducer of course)
Currently failing with:
App1Test > testHelloEndpoint() FAILED
java.lang.RuntimeException at QuarkusTestExtension.java:639
Caused by: java.lang.RuntimeException at AugmentActionImpl.java:336
Caused by: io.quarkus.builder.BuildException at Execution.java:123
Caused by: jakarta.enterprise.inject.spi.DeploymentException at BeanDeployment.java:1447
Caused by: jakarta.enterprise.inject.UnsatisfiedResolutionException at Beans.java:477
So I think I can use this for the integration test.
The test case MultiModuleKotlinProjectBuildTest.java
with the sources multi-module-kotlin-project
seems to a very good starting point for my reproducer as part of the test suite.
Sure, in case you are modifying an existing test, please make sure you are keeping the test for the original use-case. Thanks.
Did we find way for contionous build not being triggered unless we had a request coming in?
Did we find way for contionous build not being triggered unless we had a request coming in?
Gradle notifies „us“, when builds finish, including the artifacts that have been changed. Our plugin could forward the notification to the Quarkus process. The change in Quarkus-dev would be to only reload changed artifacts, but not trigger any build actions.
Hi @jmini are you still trying to fix this issue? I'm also affected and I have time help!
@clementguillot can you check if my patch https://github.com/jmini/quarkus/commit/889d63457e4017b88c8027bb4a104eca1b96cec1 would work for you?
I can try to rebase it on head of the quarkus repo.
@jmini sure, I will try on an existing multi-module Kotlin application
I was able to rebase my patch on top of origin/main
https://github.com/jmini/quarkus/commit/57183eb2bff9d091102e8b7a53902f8f2957a471
I was wondering why I could not reproduce the issue in the multi-module-kotlin-project
integration test and I found something:
Usually the root gradle project does not contains any sources, sometimes you have sections like allprojects { }
or subprojects { }
to configure each of the modules. But if you apply org.jetbrains.kotlin.jvm
to this root project, the issue can't be reproduced.
So despite the fact that there is no sources, if you add this to the root build.gradle
:
plugins {
id 'org.jetbrains.kotlin.jvm' version "1.9.10"
}
repositories {
mavenCentral()
}
The issue described here can't be reproduced.
^ I see this as a valid work-around. cc: @clementguillot
🤔 I did not exactly investigated why, but adding this in the top project modifies the behavior of the compileKotlin
task instance being or not an instance of org.jetbrains.kotlin.gradle.tasks.KotlinJvmCompile
.
Thank you @jmini for your detailed answer. I have tested on my side and I can confirm that I'm only able to reproduce issue without plugin org.jetbrains.kotlin.jvm
in root build.gradle
.
Once I globally add this plugin, everything works as expected, including hot reload. Many thanks for your help!
@jmini : Did you try if it also works when you have the kotlin jvm plugin in the root gradle but mark it as apply(false)?
The reason I'm asking is that we cannot apply this on top level because it would then apply to all submodules. This however, does not work together with Kotlin Multiplatform (which we use in our monorepo for the app of the service).
@andreas-eberle it works and it fixes couple of warnings in IntelliJ and at build-time 😉
True, it looks like it works for a normal Kotlin module. Now I have one more thing... :D It doesn't work when the submodule is a Kotlin Multiplatform module... it seems then the source dirs are different again. I then get
Execution failed for task ':server:quarkusGenerateCodeDev'.
> Cannot invoke "io.quarkus.bootstrap.workspace.ArtifactSources.getSourceDirs()" because the return value of "io.quarkus.bootstrap.workspace.WorkspaceModule$Mutable.getSources(String)" is null
Same problem here. I have a dependency on Kotlin Multiplatform module and the build is failing. Any suggestions how to workaround/fix this? I can share my project layout if interested.
Workaround till this bug exists: I have two gradle modules. One has applied kotlin.jvm
plugin, the other has applied kotlin.multiplatform
. I am linking (via symbolic link) src/main/kotlin
from the jvm module as src/main/commonMain
into the multiplatform module. This way I am effectively sharing the exactly same source code.
Describe the bug
I am in a multi-module gradle project.
:core-lib
shared lib@ApplicationScoped
):app1
quarkus app@Inject
to get the services from thecore-lib
In a real setup there are more modules (multiple shared modules, and multiple quarkus app that achieve different tasks of a more complex system)
Building the app and running it with
java -jar app1/build/quarkus-app/quarkus-run.jar
runs perfectly fine. Running with dev mode./gradlew app1:quarkusDev
is failing:(complete trace at the bottom of this message)
The error is similar to when the jandex plugin in not running in
core-lib
(from the docs: Working with multi-module gradle projects). But this is not the case here. Jandex is configured and is working. Also a work-around for https://github.com/kordamp/jandex-gradle-plugin/issues/24 is in place.But what is wired is that:
works perfectly.
Also when we build the docker image of the quarkus app, everything is working correctly.
And the issue is only present when the
core-lib
module contains kotlin code. With only java code everything works as expected.This is only a dev-mode issue.
How to Reproduce?
See repository: https://github.com/jmini/quarkus_issue35577
Output of
uname -a
orver
macOS, but it doesn't matter.
We observe the same on Linux.
Output of
java -version
GraalVM version (if different from Java)
not relevant
Quarkus version or git rev
3.3.0 (reproducible as well with lower versions)
Build tool (ie. output of
mvnw --version
orgradlew --version
)Additional information
Complete error when running `./gradlew app1:quarkusDev`
``` 2023-08-26 13:38:06,883 INFO [io.qua.dep.dev.IsolatedDevModeMain] (main) Attempting to start live reload endpoint to recover from previous Quarkus startup failure > :ap2023-08-26 13:38:07,507 ERROR [io.qua.dep.dev.IsolatedDevModeMain] (main) Failed to start quarkus: java.lang.RuntimeException: io.quarkus.builder.BuildException: Build failure: Build failed due to errors [error]: Build step io.quarkus.arc.deployment.ArcProcessor#validate threw an exception: jakarta.enterprise.inject.spi.DeploymentException: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type org.acme.core.kt.ConfigService and qualifiers [@Default] - java member: org.acme.app1.App1Controller#config - declared on CLASS bean [types=[org.acme.app1.App1Controller, java.lang.Object], qualifiers=[@Default, @Any], target=org.acme.app1.App1Controller] at io.quarkus.arc.processor.BeanDeployment.processErrors(BeanDeployment.java:1447) at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:311) at io.quarkus.arc.processor.BeanProcessor.initialize(BeanProcessor.java:158) at io.quarkus.arc.deployment.ArcProcessor.validate(ArcProcessor.java:469) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:858) at io.quarkus.builder.BuildContext.run(BuildContext.java:282) at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18) at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538) at java.base/java.lang.Thread.run(Thread.java:833) at org.jboss.threads.JBossThread.run(JBossThread.java:501) Caused by: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type org.acme.core.kt.ConfigService and qualifiers [@Default] - java member: org.acme.app1.App1Controller#config - declared on CLASS bean [types=[org.acme.app1.App1Controller, java.lang.Object], qualifiers=[@Default, @Any], target=org.acme.app1.App1Controller] at io.quarkus.arc.processor.Beans.resolveInjectionPoint(Beans.java:477) at io.quarkus.arc.processor.BeanInfo.init(BeanInfo.java:624) at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:299) ... 13 more at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:336) at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:253) at io.quarkus.runner.bootstrap.AugmentActionImpl.createInitialRuntimeApplication(AugmentActionImpl.java:60) at io.quarkus.deployment.dev.IsolatedDevModeMain.firstStart(IsolatedDevModeMain.java:82) at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:423) at io.quarkus.deployment.dev.IsolatedDevModeMain.accept(IsolatedDevModeMain.java:55) at io.quarkus.bootstrap.app.CuratedApplication.runInCl(CuratedApplication.java:138) at io.quarkus.bootstrap.app.CuratedApplication.runInAugmentClassLoader(CuratedApplication.java:93) at io.quarkus.deployment.dev.DevModeMain.start(DevModeMain.java:131) at io.quarkus.deployment.dev.DevModeMain.main(DevModeMain.java:62) Caused by: io.quarkus.builder.BuildException: Build failure: Build failed due to errors [error]: Build step io.quarkus.arc.deployment.ArcProcessor#validate threw an exception: jakarta.enterprise.inject.spi.DeploymentException: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type org.acme.core.kt.ConfigService and qualifiers [@Default] - java member: org.acme.app1.App1Controller#config - declared on CLASS bean [types=[org.acme.app1.App1Controller, java.lang.Object], qualifiers=[@Default, @Any], target=org.acme.app1.App1Controller] at io.quarkus.arc.processor.BeanDeployment.processErrors(BeanDeployment.java:1447) at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:311) at io.quarkus.arc.processor.BeanProcessor.initialize(BeanProcessor.java:158) at io.quarkus.arc.deployment.ArcProcessor.validate(ArcProcessor.java:469) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:858) at io.quarkus.builder.BuildContext.run(BuildContext.java:282) at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18) at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538) at java.base/java.lang.Thread.run(Thread.java:833) at org.jboss.threads.JBossThread.run(JBossThread.java:501) Caused by: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type org.acme.core.kt.ConfigService and qualifiers [@Default] - java member: org.acme.app1.App1Controller#config - declared on CLASS bean [types=[org.acme.app1.App1Controller, java.lang.Object], qualifiers=[@Default, @Any], target=org.acme.app1.App1Controller] at io.quarkus.arc.processor.Beans.resolveInjectionPoint(Beans.java:477) at io.quarkus.arc.processor.BeanInfo.init(BeanInfo.java:624) at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:299) ... 13 more at io.quarkus.builder.Execution.run(Execution.java:123) at io.quarkus.builder.BuildExecutionBuilder.execute(BuildExecutionBuilder.java:79) at io.quarkus.deployment.QuarkusAugmentor.run(QuarkusAugmentor.java:160) at io.quarkus.runner.bootstrap.AugmentActionImpl.runAugment(AugmentActionImpl.java:332) ... 9 more Caused by: jakarta.enterprise.inject.spi.DeploymentException: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type org.acme.core.kt.ConfigService and qualifiers [@Default] - java member: org.acme.app1.App1Controller#config - declared on CLASS bean [types=[org.acme.app1.App1Controller, java.lang.Object], qualifiers=[@Default, @Any], target=org.acme.app1.App1Controller] at io.quarkus.arc.processor.BeanDeployment.processErrors(BeanDeployment.java:1447) at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:311) at io.quarkus.arc.processor.BeanProcessor.initialize(BeanProcessor.java:158) at io.quarkus.arc.deployment.ArcProcessor.validate(ArcProcessor.java:469) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77) at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.base/java.lang.reflect.Method.invoke(Method.java:568) at io.quarkus.deployment.ExtensionLoader$3.execute(ExtensionLoader.java:858) at io.quarkus.builder.BuildContext.run(BuildContext.java:282) at org.jboss.threads.ContextHandler$1.runWith(ContextHandler.java:18) at org.jboss.threads.EnhancedQueueExecutor$Task.run(EnhancedQueueExecutor.java:2513) at org.jboss.threads.EnhancedQueueExecutor$ThreadBody.run(EnhancedQueueExecutor.java:1538) at java.base/java.lang.Thread.run(Thread.java:833) at org.jboss.threads.JBossThread.run(JBossThread.java:501) Caused by: jakarta.enterprise.inject.UnsatisfiedResolutionException: Unsatisfied dependency for type org.acme.core.kt.ConfigService and qualifiers [@Default] - java member: org.acme.app1.App1Controller#config - declared on CLASS bean [types=[org.acme.app1.App1Controller, java.lang.Object], qualifiers=[@Default, @Any], target=org.acme.app1.App1Controller] at io.quarkus.arc.processor.Beans.resolveInjectionPoint(Beans.java:477) at io.quarkus.arc.processor.BeanInfo.init(BeanInfo.java:624) at io.quarkus.arc.processor.BeanDeployment.init(BeanDeployment.java:299) ... 13 more ```I have tried to inspect the generated jandex file in
core-lib
(using a jbang script) and it looks OK (my classes and its annotation are present)Script to inspect the jandex index
```java ///usr/bin/env jbang "$0" "$@" ; exit $? //DEPS io.smallrye:jandex:3.1.2 //JAVA 17 import java.io.FileInputStream; import org.jboss.jandex.AnnotationInstance; import org.jboss.jandex.ClassInfo; import org.jboss.jandex.Index; import org.jboss.jandex.IndexReader; public class JandexMain { public static void main(String[] args) throws Exception { try (FileInputStream input = new FileInputStream("core-lib/build/resources/main/META-INF/jandex.idx")) { IndexReader reader = new IndexReader(input); Index index = reader.read(); // java.util.CollectionAlso when look at the trace logs of the Annotation Transformers (see docs), configured with:
I see only the line for the class written in java:
Nothing for the class written in Kotin, so it makes sense that it can not be injected later on.