Java extension pack does not work with Micronaut 4 applications.

sdedic commented 1 year ago

Environment

Operating System: Ubuntu Linux 22.04, KDE desktop environment
JDK version: "17.0.3.1" 2022-04-22 LTS
Visual Studio Code version: 1.79.2
Java extension version: 0.39.0

Steps To Reproduce

open the project; install gradle + java extension pack, as suggested by vscode
wait for the project loading to complete
display Problems window

Projects created for Micronaut 3.10 (3.9.x) works OK, projects targetting Micronaut 4.x fail to work properly in vscode. The sample projects are attached: demo4.zip (gradle Micronaut 4.0.1), demom4.zip (maven Micronaut 4.0.1), demo2.zip(gradle, Micronaut 3.10-SNAPSHOT -- working). During testing and switching projects, the demo4 project appeared to load OK two times, but mostly fails. The seemingly successful opens were:

open demo2, then create new window and open demo4 there. Appeared without Problems once (further attempt failed with the reported problem)
open demo2, then Close Folder, and Open Folder demo4 in the current vscode (empty) window. (again, once seemed to load OK, other attempts fail) In both successful cases, I can't say for sure the progress for "Opening Java projects" appeared at all, maybe the extension did not load fully (?)
demo2.zip -- working
demo4.zip -- fails
demom4.zip -- fails too

Logs: jdtls.log

Current Result

Rebuild All or Rebuild Project actions fail. The problems window shows the following messages:

[{
    "resource": "/space/tmp/vsc4/src/demom4",
    "owner": "_generated_diagnostic_collection_name_#2",
    "code": "0",
    "severity": 8,
    "message": "The project was not built since its build path is incomplete. Cannot find the class file for io.micronaut.inject.visitor.TypeElementVisitor. Fix the build path then try building this project",
    "source": "Java",
    "startLineNumber": 1,
    "startColumn": 1,
    "endLineNumber": 1,
    "endColumn": 1
}]
[{
    "resource": "/space/tmp/vsc4/src/demo4/src/main/java/com/example4/Application.java",
    "owner": "_generated_diagnostic_collection_name_#3",
    "code": "16777563",
    "severity": 8,
    "message": "The type io.micronaut.inject.visitor.TypeElementVisitor cannot be resolved. It is indirectly referenced from required type io.micronaut.scheduling.async.validation.AsyncTypeElementVisitor",
    "source": "Java",
    "startLineNumber": 1,
    "startColumn": 1,
    "endLineNumber": 1,
    "endColumn": 2
}]

The application fails to launch: micronaut-launch-failure

Expected Result

No problems :) or just normal code issues.

Additional Informations

Default (system) Java version:

java version "17.0.3.1" 2022-04-22 LTS
Java(TM) SE Runtime Environment (build 17.0.3.1+2-LTS-6)
Java HotSpot(TM) 64-Bit Server VM (build 17.0.3.1+2-LTS-6, mixed mode, sharing)

vscode version:

Version: 1.79.2
Commit: 695af097c7bd098fbf017ce3ac85e09bbc5dda06
Date: 2023-06-14T08:59:55.818Z
Electron: 22.5.7
Chromium: 108.0.5359.215
Node.js: 16.17.1
V8: 10.8.168.25-electron.0
OS: Linux x64 5.15.0-58-generic

Installed extensions:

redhat.java@1.20.0
VisualStudioExptTeam.intellicode-api-usage-examples@0.2.7
VisualStudioExptTeam.vscodeintellicode@1.2.30
vscjava.vscode-gradle@3.12.7
vscjava.vscode-java-debug@0.52.0
vscjava.vscode-java-dependency@0.23.0
vscjava.vscode-java-pack@0.25.12
vscjava.vscode-java-test@0.39.0
vscjava.vscode-maven@0.41.0

Suprisingly, opening the projects in Eclipse J2EE IDE shows no problems - screenshots of the Problems window:

akaroml commented 1 year ago

Looks like related to to annotation processing because of the mention TypeElementVisitor in the error log.

Is the build successful in Eclipse? If the application can be launched successfully in Eclipse, then the build should be fine. @sdedic

Would you please take a look as well? @testforstephen

sdedic commented 1 year ago

Good point: although the project demo4 opens fine (no errros in Problems after project open), Build/run actions in Eclipse behave strangely (note: I do not routinely use Eclipse, so I may need guidance in obtaining logs etc):

when Project | Build Automatically is disabled, Build Project action has (seemingly) no effect, run fails with java.lang.ClassNotFoundException: com.example4.Application.No build or bin directory is created.
when Build Automatically is enabled, bin is created on project open and app launch works.

Actually the behaviour with explicit builds (Automatic build disabled) is quite erratic. I tried to close/open project(s) and restart Eclipse IDE

sometimes bin is created on project open. Sometimes not.
sometimes build is created on IDE start or project open, with just resources. In one test run, neither build or bin appeared no matter what I've tried (Build Project, Build All, Clean + Build, ...)
sometimes Build Project produces classes into bin folder, but sometimes not - there are just resources in build No errors are logged in Problems window or Error Log. I can't find a pattern in the behaviour.

Note: I mentioned Eclipse just because I thought the compilation + problems engine is mostly the same as in vscode lang server for java, so I was surprised with the behaviour difference.

testforstephen commented 1 year ago

This is a known issue of ECJ compiler, see https://github.com/eclipse-jdt/eclipse.jdt.core/issues/543.

ECJ requires all indirect dependencies in the build path when compiling a project. But in your sample micronaut project, there is a dependency micronaut-context-4.0.1.jar (io.micronaut.scheduling.async.validation.AsyncTypeElementVisitor) that references an implicit dependency micronaut-core-processor-4.0.1.jar (io.micronaut.inject.visitor.TypeElementVisitor), which cannot be found in the project build path.

The workaround is to explicitly add the missing dependency micronaut-core-processor to the project dependencies list.

implementation('io.micronaut:micronaut-core-processor:4.0.1')

sdedic commented 1 year ago

@testforstephen hm ! micronaut-context currently lists micronaut-core-processor just as compileOnly dependency (which is not exported to consumers ?) - would it help if it was present also as the micronaut-context s annotationProcessor or compileOnlyApi dependency ?

Anyway, gradle buildsys is apparently able to build the project, it seems to collect "somehow" the dependencies properly - shouldn't be the project info on par with that ?

sdedic commented 1 year ago

To anwer myself. No: annotationProcessor is not transitive, and compileOnlyApi would leak the core-processor to application's compile classpath (as opposed to be only available to APs as a runtime dependency).

The suggested workaround is not 'correct' as well, as it leaks internal AP dependencies to the consuming project(s).

BTW the issue affects also Maven (see demom4) which has a different annotation CP construction rules (and also does not work when imported to vscode)

graemerocher commented 1 year ago

this is unfortunate behaviour of Eclipse JDT. No wonder nobody it is rarely used when annotation processors are used extensively.

I have tried to implement a workaround https://github.com/micronaut-projects/micronaut-core/pull/9634 in Micronaut, but this may come up again as this is clearly broken behaviour of the IDE

graemerocher commented 1 year ago

So 4.0.2 is out and the build problem is gone, but now it doesn't look like annotation processors are running at all

testforstephen commented 1 year ago

So 4.0.2 is out and the build problem is gone, but now it doesn't look like annotation processors are running at all

Here is the exception log for this failure.

!ENTRY org.eclipse.jdt.apt.pluggable.core 4 1 2023-08-02 14:39:18.047
!MESSAGE Exception thrown by Java annotation processor io.micronaut.annotation.processing.BeanDefinitionInjectProcessor@237cb634
!STACK 0
java.lang.Exception: org.eclipse.jdt.internal.compiler.problem.AbortCompilation: 
    at org.eclipse.jdt.internal.compiler.apt.dispatch.RoundDispatcher.handleProcessor(RoundDispatcher.java:172)
    at org.eclipse.jdt.internal.compiler.apt.dispatch.RoundDispatcher.round(RoundDispatcher.java:124)
    at org.eclipse.jdt.internal.compiler.apt.dispatch.BaseAnnotationProcessorManager.processAnnotations(BaseAnnotationProcessorManager.java:172)
    at org.eclipse.jdt.internal.apt.pluggable.core.dispatch.IdeAnnotationProcessorManager.processAnnotations(IdeAnnotationProcessorManager.java:138)
    at org.eclipse.jdt.internal.compiler.Compiler.processAnnotations(Compiler.java:953)
    at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:450)
    at org.eclipse.jdt.internal.compiler.Compiler.compile(Compiler.java:426)
    at org.eclipse.jdt.internal.core.builder.AbstractImageBuilder.compile(AbstractImageBuilder.java:406)
    at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.compile(BatchImageBuilder.java:214)
    at org.eclipse.jdt.internal.core.builder.AbstractImageBuilder.compile(AbstractImageBuilder.java:338)
    at org.eclipse.jdt.internal.core.builder.BatchImageBuilder.build(BatchImageBuilder.java:79)
    at org.eclipse.jdt.internal.core.builder.JavaBuilder.buildAll(JavaBuilder.java:276)
    at org.eclipse.jdt.internal.core.builder.JavaBuilder.build(JavaBuilder.java:188)
    at org.eclipse.core.internal.events.BuildManager$2.run(BuildManager.java:1023)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:247)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:303)
    at org.eclipse.core.internal.events.BuildManager$1.run(BuildManager.java:392)
    at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:45)
    at org.eclipse.core.internal.events.BuildManager.basicBuild(BuildManager.java:395)
    at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:506)
    at org.eclipse.core.internal.events.BuildManager.basicBuildLoop(BuildManager.java:454)
    at org.eclipse.core.internal.events.BuildManager.build(BuildManager.java:536)
    at org.eclipse.core.internal.resources.Workspace.buildInternal(Workspace.java:524)
    at org.eclipse.core.internal.resources.Workspace.build(Workspace.java:413)
    at org.eclipse.jdt.ls.core.internal.handlers.BuildWorkspaceHandler.buildWorkspace(BuildWorkspaceHandler.java:63)
    at org.eclipse.jdt.ls.core.internal.handlers.JDTLanguageServer.lambda$27(JDTLanguageServer.java:966)
    at org.eclipse.jdt.ls.core.internal.handlers.JDTLanguageServer.lambda$56(JDTLanguageServer.java:1166)
    at java.base/java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:646)
    at java.base/java.util.concurrent.CompletableFuture$Completion.exec(CompletableFuture.java:483)
    at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
    at java.base/java.util.concurrent.ForkJoinPool$WorkQueue.topLevelExec(ForkJoinPool.java:1182)
    at java.base/java.util.concurrent.ForkJoinPool.scan(ForkJoinPool.java:1655)
    at java.base/java.util.concurrent.ForkJoinPool.runWorker(ForkJoinPool.java:1622)
    at java.base/java.util.concurrent.ForkJoinWorkerThread.run(ForkJoinWorkerThread.java:165)
Caused by: org.eclipse.jdt.internal.compiler.problem.AbortCompilation: 
    at org.eclipse.jdt.internal.core.builder.NameEnvironment.findClass(NameEnvironment.java:521)
    at org.eclipse.jdt.internal.core.builder.NameEnvironment.findType(NameEnvironment.java:591)
    at org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.lambda$0(LookupEnvironment.java:240)
    at org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForTypeFromModules(LookupEnvironment.java:375)
    at org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.askForType(LookupEnvironment.java:239)
    at org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.getType(LookupEnvironment.java:1735)
    at org.eclipse.jdt.internal.compiler.lookup.LookupEnvironment.getType(LookupEnvironment.java:1704)
    at org.eclipse.jdt.internal.compiler.apt.model.ElementsImpl9.getTypeElement(ElementsImpl9.java:100)
    at org.eclipse.jdt.internal.compiler.apt.model.ElementsImpl9.getTypeElement(ElementsImpl9.java:72)
    at io.micronaut.annotation.processing.visitor.JavaVisitorContext.getClassElement(JavaVisitorContext.java:191)
    at io.micronaut.annotation.processing.visitor.JavaClassElement.getAccessibleStaticCreators(JavaClassElement.java:612)
    at io.micronaut.inject.ast.ClassElement.findStaticCreator(ClassElement.java:295)
    at io.micronaut.inject.ast.ClassElement.getPrimaryConstructor(ClassElement.java:230)
    at io.micronaut.annotation.processing.visitor.JavaClassElement.getPrimaryConstructor(JavaClassElement.java:603)
    at io.micronaut.inject.processing.DeclaredBeanElementCreator.createBeanDefinitionVisitor(DeclaredBeanElementCreator.java:101)
    at io.micronaut.inject.processing.DeclaredBeanElementCreator.buildInternal(DeclaredBeanElementCreator.java:82)
    at io.micronaut.inject.processing.AbstractBeanElementCreator.build(AbstractBeanElementCreator.java:72)
    at io.micronaut.annotation.processing.BeanDefinitionInjectProcessor.process(BeanDefinitionInjectProcessor.java:210)
    at org.eclipse.jdt.internal.compiler.apt.dispatch.RoundDispatcher.handleProcessor(RoundDispatcher.java:142)
    ... 34 more

Looks like the annotation processors used in this sample project do not generate new source files but manipulate the existing source file Demo4Test in memory and affect the generated class files. I'm not an expert of Java annotation processing, I'm not sure if this is allowed by spec.

ECJ compiler will abort the compilation when it detects some source files have been changed during compilation. In case you're interested to investigate the compiler further, here is the ECJ compiler code that throws the exception.

graemerocher commented 1 year ago

@testforstephen we do not manipulate the AST at all in Micronaut, the call here causing the issue is a simple lookup by class name:

io.micronaut.annotation.processing.visitor.JavaVisitorContext.getClassElement(JavaVisitorContext.java:191)

All it does is call Element.getTypeElement(..)

    public Optional<ClassElement> getClassElement(String name, ElementAnnotationMetadataFactory annotationMetadataFactory) {
        TypeElement typeElement = elements.getTypeElement(name);
        if (typeElement == null) {
            // maybe inner class?
            typeElement = elements.getTypeElement(name.replace('$', '.'));
        }
        return Optional.ofNullable(typeElement)
            .map(typeElement1 -> elementFactory.newClassElement(typeElement1, annotationMetadataFactory));
    }

There are no issues with with this with javac. That said we will look if we can place yet another workaround for Eclipse JDT into the codebase.

graemerocher commented 1 year ago

btw how do you get to those logs?

graemerocher commented 1 year ago

I pushed another workaround for this bug in Eclipse JDT https://github.com/micronaut-projects/micronaut-core/pull/9660

sdedic commented 1 year ago

Here is a more simple testcase for this bug. maven-ap contains just a simple AP implementation. You need to mvn install it first, then open maven-ap2 in vscode. Recompile the project (works fine), then make a change in .java file and save. CompilationAborted is rendered as a problem in the editor. The processor does not 'change' anything, just generates new sources.

ap-testcase.zip

I suggest to upgrade the bug description, as the bug potentially affects way more APs than just Micronaut's one.

testforstephen commented 1 year ago

btw how do you get to those logs?

F1 -> Java: Open Java Language Server Log File

snjeza commented 1 year ago

Here is a more simple testcase for this bug. maven-ap contains just a simple AP implementation. You need to mvn install it first, then open maven-ap2 in vscode. Recompile the project (works fine), then make a change in .java file and save. CompilationAborted is rendered as a problem in the editor. The processor does not 'change' anything, just generates new sources.

A related Eclipse bug - https://bugs.eclipse.org/bugs/show_bug.cgi?id=547970

sdedic commented 1 year ago

Note though that this ticket actually talks about 2 different issues:

ECJ issue with annotation processors (ttps://bugs.eclipse.org/bugs/show_bug.cgi?id=547970
Buildship(?) classpath construction issue; also see https://github.com/redhat-developer/vscode-java/issues/2123 and https://github.com/redhat-developer/vscode-java/issues/1595, ttps://bugs.eclipse.org/bugs/show_bug.cgi?id=547970. Note that maven, gradle are both able to somehow construct correct classpath for the compilation from the project-provided info.

redhat-developer / vscode-java