EE 11 Platform TCK refactoring roadmap

scottmarlow commented 2 years ago

Continue Platform TCK refactoring using the https://github.com/eclipse-ee4j/jakartaee-tck/tree/tckrefactor branch as per https://github.com/eclipse-ee4j/jakartaee-platform/issues/554.

Jakarata EE11 TCK Release Project

Guidelines:

Completing refactoring of the SE mode TCKs is required for the subset of technologies that the Platform TCK generates Standalone TCKs for, these are important to complete in time for the wave of EE 11 Spec ballots which will be blocked until needed TCKs are complete.
Completing the refactoring of the EE mode tests is required for Full Platform + Web Profile profile ballots to start. Note that the TCK Process rules must be followed with regard to when test changes can be made (pending outcome specification committee issue 63).

[x] Sync https://github.com/eclipse-ee4j/jakartaee-tck/tree/tckrefactor with master (EE 10) changes already done via #1124.
[x] #1128
[ ] #1130
[x] Move contents of https://github.com/eclipse-ee4j/jakartaee-tck/tree/tckrefactor/src/com/sun/ts/tests to respective new root folders (ejb30 el jsonb jta servlet websocket).
[ ] #1141

Bootstrap Tasks

[x] https://github.com/jakartaee/platform-tck/issues/1349

Organizational/Structural Tasks

Porting Tasks

[ ] Refactor appclient (EE mode) tests.
[ ] Refactor assembly (EE mode) tests.
[ ] Refactor connector (EE mode) tests.
[ ] Refactor ejb32 (EE mode) tests.
[ ] Refactor ejb30 (EE mode) tests.
[ ] Refactor ejb (EE mode) tests.
[ ] Refactor el (EE mode) tests.
[ ] Refactor integration (EE mode) tests.
[ ] Refactor jacc (EE mode) tests.
[x] Refactor javaee (EE mode) tests.
[ ] Refactor javamail (EE mode) tests.
[x] #1156
[ ] Refactor jaxws (EE mode) tests.
[ ] Refactor jdbc (EE mode) tests.
[ ] Refactor jms (EE mode) tests.
[ ] Refactor jpa (EE mode) tests.
[ ] Refactor jsonp (EE mode) tests.
[ ] Refactor jsonb (EE mode) tests.
[x] Refactor jsp (EE mode) tests.
[x] Refactor jstl (EE mode) tests.
[ ] Refactor jta (EE mode) tests.
[ ] Refactor jws (EE mode) tests.
[ ] Refactor saaj (EE mode) tests.
[x] Refactor servlet tests (https://github.com/eclipse-ee4j/jakartaee-tck/pull/912).
[ ] Refactor sql (EE mode) tests.
[ ] Refactor websocket (EE mode) tests.
[ ] Refactor xa (EE mode) tests.
[ ] Refactor Signature test Driver and recording of new Signature for both EE + SE + Standalone mode.
[ ] Create milestone 1 release with EE mode support (optionally with SE mode as well).
[ ] Introduce (active/strong) dependencies/assertion between EE 11 Specifications and TCK tests with active reporting (for new EE 11 SPEC features).
[ ] Introduce common TCK result reporting.
[ ] Refactor more EE mode tests for each test bucket mentioned above.

Other Tasks

[ ] Add some SE tests to connector
[x] Add some SE mode tests to el
[ ] Add some SE mode tests to jacc
[ ] Add some SE + EE tests to jaxws
[ ] Add some SE mode tests to jms
[x] Add some SE mode tests to jpa
[x] Add some Standalone tests to jsp
[ ] Add some SE + EE tests to jstl
[ ] Add some SE + EE tests to jta
[ ] Add some SE mode tests to saaj
[x] Add some SE + EE tests to servlet
[ ] Add some SE + EE tests to websocket
[ ] Create milestone 2 release with (some) EE mode support + (some) SE mode.
[ ] Complete the refactoring of TCK tests.
[ ] Update the Signature testing using EE 10 SPEC APIs.
[ ] Create milestone 3 release (EE 10 implementations can still be used for testing). This is the last EE 10 based milestone.
[ ] Apply outcome of Fix handling of Specification integration TCKs.
[ ] Add tests to cover difficult to implement EE 11 Spec features (other tests may also be added). Confirm that each Specification team has added sufficient tests.
[ ] Update the documentation for the various TCKs for the milestone 4 release.
[ ] Create milestone 4 release for the first wave of Specification TCKs.
[ ] After milestone 4 release is validated by first wave of Spec projects completing their Ballots (with milestone 4 TCKs), release final first wave of TCKs reflecting their respective EE 11 Specifications requirements (with release done however TCKs are released for EE 11, maybe via Maven or current Eclipse downloads).
[ ] Repeat ^ for milestone 5 with second wave of Spec projects (will fill out the details soon). More milestones to be added here...

scottmarlow commented 2 years ago

Removed guideline jakartaee/specification-committee#2:

Completing the Standalone TCKs is required for the subset of technologies that the Platform TCK generates Standalone TCKs for, these are important to complete in time for the wave of EE 11 Spec ballots which will be blocked until needed Standalone TCKs are complete. Standalone TCKs use more than Java SE technologies for testing which can include other EE specifications.

gurunrao commented 1 year ago

TCK refactor module Owners and comments:

appclient
assembly
connector
ejb32
ejb30
ejb
el : @alwin-joseph
integration
jacc
javaee: doesn't have test case and not related to Specification.
javamail
jaxrs: @alwin-joseph
jaxws: @gurunrao
jdbc
jms
jpa: @gurunrao
jsonp
jsonb
jsp: @alwin-joseph
jstl
jta :
jws: Part of Standalone TCK and Platform TCK, shall we retain the code without refactor and maintain JWS as part of Standalone TCK?
saaj: @gurunrao
servlet: @olamy
sql: common SQL scripts used for test setup.
websocket: @gurunrao
xa

scottmarlow commented 1 year ago

FYI https://gist.github.com/scottmarlow/cc7b689ed3ddadf1460bcea5c2f7518a is a backup of the description content before the May 3, 2023 update.

scottmarlow commented 1 year ago

jaxws: Standalone TCK, shall we retain the code without refactor?

jws: Part of Standalone TCK and Platform TCK, shall we retain the code without refactor and maintain JWS as part of Standalone TCK?

saaj: Standalone TCK, shall we retain the code without refactor?

Good questions, the source for these TCKs is part of the https://github.com/jakartaee/platform-tck/tree/10.0.x branch. Regardless of what we choose to do for the EE 11 release, the source will remain in the 10.0.x branch. I'm thinking that the relevant SPEC API projects should copy the relevant Standalone TCK source (either from refactor branch if they want the Maven changes or from the 10.0.x if they prefer to use Ant in future updates to these Specification TCKs).

jakartaee / platform-tck