TCK Support for JDK 12+

[brideck]

A number of updates are going to be needed in the signature tests in order to support JDK versions greater than 11. Currently, the api/signaturetest fails because it only explicitly supports JDK 8 & 11. 183 of the sigtests in the rest of the TCK fail because of changes to java.lang.Enum in JDK 12+.

example failure:

Added Nested Classes

javasoft.sqe.tests.localscoping.localscoping00102m1.ItemValues: nested public final static java.lang.Enum$EnumDesc

Added Superclasses or Superinterfaces

javasoft.sqe.tests.localscoping.localscoping00102m1.ItemValues: interface java.lang.constant.Constable

Added Methods

javasoft.sqe.tests.localscoping.localscoping00102m1.ItemValues: method public final java.util.Optional<java.lang.Enum$EnumDesc> java.lang.Enum.describeConstable()

A number of changes was made in the platform TCK for EE 9.1 in order to streamline and futureproof the signature tests there. Similar changes would need to happen here. The platform TCK now uses an updated sigtest JAR and made a number of changes to sig files and driver classes.

Details of the changes in the platform TCK can be seen in these 2 commits: https://github.com/eclipse-ee4j/jakartaee-tck/commit/605de32c7ec214e7f5fe1a640905a1bf766d49f9 https://github.com/eclipse-ee4j/jakartaee-tck/commit/888c12cb12b2596d4c74058c5f470346d9eed558

[scottmarlow]

@lukasj For making the changes to address this, I think we need to switch to the newer sigtest jar (can copy from Platform TCK), delete the JDK11 signature map files (switch to only reference the JDK8 signature map files) and pass in a custom list of JDK classes to exclude from signature testing based on what is actually referenced for jaxb.

The same changes for the Platform TCK were made via the 2 commits that Brian mentioned in the description.

The trick for picking the JDK classes to ignore is based on which JDK classes cause JAXB TCK signature test failures (e.g. java.lang.Enum would be one of them).

Should we create a new TCK challenge issue or add the challenge label to this issue?

[lukasj]

can someone just create a PR addressing this? If not, I'm pretty sure, this gets addressed at some point for EE10. Or is this something requiring immediate attention?

[scottmarlow]

@lukasj The attached 183 jtr files show that the same signature test failures repeating jaxbjtrfiles.zip. Yes, I think that someone can create a PR for addressing this for EE 10.

For Jakarta EE 9.1 certification requests that include JAXB test results from running on Java SE 17, they will show the same signature test failures repeatedly, so I think that a challenge is needed for the signature test failures. Do you have a preference for whether we add the challenge label to this issue or create a new issue for the challenge?

The only thing that we could do for the XML Binding 3.0 release is to accept the challenge as allowing the workaround that the signature test failures are expected for the mentioned JDK classes when running on JDK 12+.

The alternative could be to create another Jakarta EE 9.something service release.

[kwsutter]

My question (request) is whether these signature test updates should be done in a 9.1.1 TCK update instead of waiting for EE 10. Since we indicated that any Java runtime from 11 forward could be used to test 9.1, it seems that we should correct problems via a service release instead of waiting for 10. I don't know the extent of the required changes and the impact to past testing efforts. So, we need to understand that before deciding which direction to go. But, we have 180+ signature tests that are currently failing. That's a large number of tests to "ignore" if testing with a runtime beyond java 11. It would be much cleaner to provide a 9.1.1 update -- if it's possible.

[gurunrao]

@lukasj @scottmarlow @kwsutter - I will start work on the issue based on Platform TCK commits: eclipse-ee4j/jakartaee-tck@605de32 eclipse-ee4j/jakartaee-tck@888c12c

Since JAXB TCK is not part of platform TCK, if we release JAXB 3.0.2, it would be sufficient.

[lukasj]

I'd personally strongly object against relying on/fully supporting JDK 17 before it goes final as the amount of work may vary from "do it ones" to "fix it for every new JDK 17 build", even though JDK 17 has already entered ramp down phase one. ... anyway, is it allowed to use non-final Java SE build/version for certifications?

[scottmarlow]

Sorry, I missed this other .jtr test result file that just fails because of result: Failed. Unsupported Java version: 17-ea failure. SignatureTest.jtr.zip

[scottmarlow]

I'd personally strongly object against relying on/fully supporting JDK 17 before it goes final as the amount of work may vary from "do it ones" to "fix it for every new JDK 17 build", even though JDK 17 has already entered ramp down phase one.

I'm running with JDK17 but I don't think Brian Decker was. So I think it can also be recreated with JDK12+ as Brian mentioned above.

Do you need me to create a separate challenge issue or can this issue represent the TCK challenge (e.g. add challenge label?)

... anyway, is it allowed to use non-final Java SE build/version for certifications?

I am not aware of a rule against it. I would expect the EE implementation to self validate that they still pass with the final Java SE version later.

[bstansberry]

The SE 17 aspect is interesting to WildFly (Preview) because we're evaluating where we stand with 17 to get ready for when it goes GA. But before that happens I'd like to be able to certify a WildFly Preview release as compatible when running SE 16. We see the same results when running the JAXB TCK with 16 or 17-ea.

[brideck]

I'd personally strongly object against relying on/fully supporting JDK 17 before it goes final as the amount of work may vary from "do it ones" to "fix it for every new JDK 17 build", even though JDK 17 has already entered ramp down phase one.

I'm running with JDK17 but I don't think Brian Decker was. So I think it can also be recreated with JDK12+ as Brian mentioned above.

That's right. I see these same failures with both JDK 15 & 16. The content of the failures points to changes in java.lang.Enum that were introduced in 12. Nothing that we've seen so far would in any way be sensitive to build-by-build differences as 17 approaches a release.

[lukasj]

Ah, I missed the change in wording in the platform spec which up to v9 defined fixed range of Java SE versions which must be supported to an open range starting at Java SE 8 in EE 9.1, in particular this:

This specification requires that containers provide a Java Compatible™ runtime environment, as defined by the Java Platform, Standard Edition, v8 specification (Java SE).

was changed to:

This specification requires that containers support execution in a Java™ runtime environment, as defined by the Java Platform, Standard Edition specification, v8 or later (Java SE 8 or later).

My bad, sorry. That was, IMHO, unfortunate change - or did anyone tried to certify/verify TCKs on 9 and/or 10 which are clearly later than 8? Like with 17-eas, it is not forbidden to use them, so I'd expect that even signature tests do all pass there. To be fair, it is unlikely there exists a product which is willing to support 9 and/or 10, but that still does not mean, that TCKs should not be usable on these Java SE versions - Platform spec allows everything from 8, so TCKs should be able to follow that, whatever that means.

@gurunrao Also take a look at jaf and mail TCKs, it's likely they suffer from the same problem. Thanks!

[scottmarlow]

My question (request) is whether these signature test updates should be done in a 9.1.1 TCK update instead of waiting for EE 10. Since we indicated that any Java runtime from 11 forward could be used to test 9.1, it seems that we should correct problems via a service release instead of waiting for 10. I don't know the extent of the required changes and the impact to past testing efforts. So, we need to understand that before deciding which direction to go. But, we have 180+ signature tests that are currently failing. That's a large number of tests to "ignore" if testing with a runtime beyond java 11. It would be much cleaner to provide a 9.1.1 update -- if it's possible.

Can we do a service release of just the impacted Specifications { JAXB, JAF, JAVAMAIL }?

We do not yet have a challenge for JAF + JAVAMAIL but I suppose via inspection we should be able to determine if they are impacted.

[lukasj]

Can we do a service release of just the impacted Specifications { JAXB, JAF, JAVAMAIL }?

You mean TCKs only, right? Xmlb spec SR is pending review since end of March because there is no process to produce spec doc update.., so unless that part skipped (again), the full SR release may take some, very likely non-trivial amount of time

We do not yet have a challenge for JAF + JAVAMAIL but I suppose via inspection we should be able to determine if they are impacted.

(Jakarta) Mail and JavaMail are two different projects - the former is successor of the latter. To avoid confusion - jakarta mail is ment here

[scottmarlow]

You mean TCKs only, right? Xmlb spec SR is pending review since end of March because there is no process to produce spec doc update.., so unless that part skipped (again), the full SR release may take some, very likely non-trivial amount of time

Thanks, yes, I meant the TCKs for the specifications.

[gurunrao]

PR for ^ issue - https://github.com/eclipse-ee4j/jaxb-tck/pull/58

[scottmarlow]

@gurunrao is there a TCK build somewhere that I can download to verify the fix with my local testing?

[gurunrao]

@scottmarlow - please use build - https://ci.eclipse.org/jakartaee-tck/job/jaxb-tck/job/master/103/artifact/bundles/xml-binding-tck-3.0.2.zip

[scottmarlow]

I'll try https://ci.eclipse.org/jakartaee-tck/job/jaxb-tck/job/master/108/artifact/bundles/xml-binding-tck-3.0.2.zip since build 103 got removed :-)

[scottmarlow]

I verified that with OpenJDK17 (EA) the https://ci.eclipse.org/jakartaee-tck/job/jaxb-tck/job/master/108/artifact/bundles/xml-binding-tck-3.0.2.zip tests passed! Excellent work with the change!

java -version
openjdk version "17-ea" 2021-09-14
OpenJDK Runtime Environment (build 17-ea+24-2164)
OpenJDK 64-Bit Server VM (build 17-ea+24-2164, mixed mode, sharing)

[gurunrao]

JAXB 3.0.2 promotion and certification has been completed. CI Certification builds:

• https://ci.eclipse.org/jakartaee-tck/job/eftl-jaxbtck-certification/15/
• https://ci.eclipse.org/jakartaee-tck/job/eftl-jaxbtck-certification/16/

JAXB 3.0.2 promoted Bundle can be found at https://download.eclipse.org/ee4j/jakartaee-tck/jakartaee9-eftl/promoted/jakarta-xml-binding-tck-3.0.2.zip