PDE errors on all bundles that use system packages since I20240412-1800

[iloveeclipse]

Since https://download.eclipse.org/eclipse/downloads/drops4/I20240412-1800/ we have PDE errors reported on all bundles that make use of system packages, like on jface:

Description Resource    Path    Location    Type
Neither an available bundle nor the associated JRE export package 'javax.xml.parsers'   MANIFEST.MF /org.eclipse.jface/META-INF line 42 Plug-in Problem
Neither an available bundle nor the associated JRE export package 'org.w3c.dom' MANIFEST.MF /org.eclipse.jface/META-INF line 44 Plug-in Problem
Neither an available bundle nor the associated JRE export package 'org.xml.sax' MANIFEST.MF /org.eclipse.jface/META-INF line 45 Plug-in Problem

This is a regression from https://github.com/eclipse-jdt/eclipse.jdt.debug/pull/426, reverting it fixes the issue.

TL;DR: at some place in PDE broken 23 EE is picked up as compatible with Java 17, and it since it has zero system packages, we have trouble in resolving system package dependencies in PDE.

Looks like adding "broken" JavaSE-23 EE profile (it doesn't have proper compliance set in org.eclipse.jdt.internal.launching.environments.EnvironmentsManager.getExecutionEnvironmentCompliance(IExecutionEnvironment)) confuses PDE. I haven't found exact place yet, but it is most likely due the fact that EE's are sorted by compliance and since JavaSE-23 EE has 1.3 compliance it seem to be picked before any other "Java 17 compatible" EE and because of org.eclipse.pde.internal.build.site.PDEState.querySystemPackages(IExecutionEnvironment, Properties) code no system packages can be found if Java 23 JVM is not "installed" as JVM in Eclipse.

This is most likely also the reason for 3 new PDE test failures in ClasspathResolutionTest that needs proper resolved system packages, see https://download.eclipse.org/eclipse/downloads/drops4/I20240412-1800/testresults/html/org.eclipse.pde.ui.tests_ep432I-unit-win32-java17_win32.win32.x86_64_17.html

[iloveeclipse]

Adding 23 ea JVM install from https://jdk.java.net/23/ to Eclipse "workarounds" the issue locally.

[HannesWell]

I haven't found exact place yet, but it is most likely due the fact that EE's are sorted by compliance and since JavaSE-23 EE has 1.3 compliance it seem to be picked before any other "Java 17 compatible" EE and because of org.eclipse.pde.internal.build.site.PDEState.querySystemPackages(IExecutionEnvironment, Properties) code no system packages can be found if Java 23 JVM is not "installed" as JVM in Eclipse.

Can you check what the Execution Environments preference pages shows as compatible JREs for Java-17? Actually if there is no JRE compatible to an EE in the workspace, a requirement to that EE should not resolve. But I have to check if that's already happening or an artifact of the broken JDT metadata (if I understand your report correctly something is missing there?)

[iloveeclipse]

The main reason is the inconsistency of the added EE because the code that is supposed to return Java compliance in org.eclipse.jdt.internal.launching.environments.EnvironmentsManager.getExecutionEnvironmentCompliance(IExecutionEnvironment) returns wrong compliance (1.3 instead of 23), because the code in EnvironmentsManager was not updated because there is no JavaCore.VERSION_23 added yet. So org.eclipse.jdt.launching is "out-of-sync" and ahead JDT with Java 23 support. This inconsistency isn't expected by all the code that relies on Java execution environments in one or another way.

[iloveeclipse]

Because of broken compliance Java 23 EE is found first as compliant to Java 17 (it should be Java 17 of course) but it has zero system packages as long as Java 23 JDK is not installed (system packages are queried from installation).

[akurtakov]

I will try adding JavaCore.VERSION_23 and use it on Monday. For Java 22 this constant was added on master so should be fine to do it again.

[iloveeclipse]

Because of broken compliance Java 23 EE is found first as compliant to Java 17

I've "patched" org.eclipse.jdt.internal.launching.environments.EnvironmentsManager.getExecutionEnvironmentCompliance(IExecutionEnvironment) to return 23 compliance for Java 23 and that fixed the problem.

However I haven't investigated / analyzed which side effects will appear after properly adding JavaCore.VERSION_23 constant with all the related code that need to be changed (looking on https://github.com/eclipse-jdt/eclipse.jdt.core/commit/31ef12dd145f22210d90dc53596f0799d420482b or https://github.com/eclipse-jdt/eclipse.jdt.core/commit/9b2546e0251cac12beb116074445391c675ab4f7)

[laeubi]

I must confess I not fully understand why its an PDE issue when JDT returns wrong compliance or did I got the explanation wrong? So what exactly is supposed to change at PDE?

[akurtakov]

I have problem reproducing the issue. Here are my JVMs:

Here is my Java 17 EE pref page:

And jface bundle resolving all deps:

So what do I have different that prevents me from reproducing?

[merks]

I thought you needed a Java 23 JRE. But @iloveeclipse knows best...

[akurtakov]

I have actually removed Java 23 from Eclipse due to https://github.com/eclipse-pde/eclipse.pde/issues/1227#issuecomment-2053979268 which made me believe Java 23 installed makes issue goes away.

[iloveeclipse]

So what do I have different that prevents me from reproducing?

Wrong SDK build? No clean build? No idea. Put breakpoint in getExecutionEnvironmentCompliance() https://github.com/eclipse-jdt/eclipse.jdt.debug/blob/61a610fc99b4b5bff929515d6fb9a7998d5b618c/org.eclipse.jdt.launching/launching/org/eclipse/jdt/internal/launching/environments/EnvironmentsManager.java#L246 it should hit for Java 23 EE.

I see the problem on my both Linux/Windows workspaces, with fully different JDK's installed. The problem disappears after adding Java 23 SDK and appears again if removing it.

[iloveeclipse]

I guess if you add Java 7 or 8 installation it may show the problem. I see you have only Java's from 11 upwards, but both my workspaces have Java 7 & 8 installed. I believe somewhere in PDE lurks some code that "mimics" older (non-modular) Java version by looking at a modular one and that one prevents you to see the problem.

[akurtakov]

Installed Java 7 and 8 from SDKman and added them to the workbench no change. As adding constants to master has been done once (https://github.com/eclipse-jdt/eclipse.jdt.core/pull/1756/files) I'll proceed with adding constants for Java 23 now.

[iloveeclipse]

Installed Java 7 and 8 from SDKman and added them to the workbench no change.

May be restart is needed. If I remember right, some PDE code runs at startup.

[iloveeclipse]

BTW, you can check if org.eclipse.pde.core.tests.internal.classpath.ClasspathResolutionTest fails for you (it does for me). After the wrong compliance is fixed, it start to work again.

[akurtakov]

@iloveeclipse I can confirm that the test fails with I20240414-1800 and works after fixing the wrong compliance - this makes me believe that we are on the right path even if I can't reproduce the issue locally. Regarding restarts, for every change in jvm lists I tried in running workbench to add remove, restart and try again.

[iloveeclipse]

I can't reproduce the issue locally.

Must be yet another "order" issue in PDE similar to what we've seen in https://github.com/eclipse-pde/eclipse.pde/pull/1093.

[iloveeclipse]

With I20240416-0300 build I don't see any errors in the workspace anymore.