search

jakartaee / platform-tck

Jakartaee-tck
Other
126 stars 105 forks source link

Need one way to setup/cleanup tests that works for both Platform TCK component TCKs + EE test mode #1420

Closed scottmarlow closed 3 weeks ago

scottmarlow commented 1 month ago

For EE testing, we need to support (https://github.com/jakartaee/platform-tck/blob/main/common/src/main/java/com/sun/ts/lib/harness/EETest.java) test setup/cleanup.

This issue is a compromise to support both the new @BeforeEach/AfterEach way for component TCKs but still allow the old way for EE Test setup/cleanup.

scottmarlow commented 1 month ago

One concern with using @BeforeEach/AfterEach for EE testing is ensuring that the BeforeEach method is not called too soon and that the AfterEach method is also called at the right time.

scottmarlow commented 1 month ago

To identify the TCK tests that need changes:

grep -rli org.junit.jupiter.api.BeforeEach grep -rli org.junit.jupiter.api.AfterEach

Specifications for which the TCK tests need updates (to BeforeEach/AfterEach methods):

The changes needed are to support EE testing:

  1. Each BeforeEach method needs to check a system property indicates Jakarta EE deployment is being used and that the @BeforeEach public void _setup() throws Exception { } method should return right away. Otherwise, the public void setup() throws Exception should be called. The EE testing layer will directly call the public void setup() throws Exception method which doesn't check for the EE system property.
  2. Do the same for each AfterEach method.

The test changes should be done in a way that doesn't change any test logic in already balloted component specifications.

Note that there are 566 references to BeforeEach as per grep -rli org.junit.jupiter.api.BeforeEach --include=*.java| wc -l Note that there are 358 references to AfterEach as per grep -rli org.junit.jupiter.api.AfterEach --include=*.java| wc -l

scottmarlow commented 1 month ago

Note that some tests still use https://github.com/jakartaee/platform-tck/blob/main/common/src/main/java/com/sun/ts/lib/harness/SetupMethod.java annotation to mark the test setup method and https://github.com/jakartaee/platform-tck/blob/main/common/src/main/java/com/sun/ts/lib/harness/CleanupMethod.java to mark the test cleanup method.

But these tests are not currently being tested yet.

gurunrao commented 1 month ago

To identify the TCK tests that need changes:

grep -rli org.junit.jupiter.api.BeforeEach grep -rli org.junit.jupiter.api.AfterEach

Specifications for which the TCK tests need updates (to BeforeEach/AfterEach methods):

* https://jakarta.ee/specifications/expression-language/6.0/ which is new for EE 11

* https://jakarta.ee/specifications/persistence/3.2 which is new for EE 11

* https://jakarta.ee/specifications/websocket/2.2 which is new for EE 11

* https://jakarta.ee/specifications/pages/4.0 which is new for EE 11

The changes needed are to support EE testing:

1. Each BeforeEach method needs to check a system property indicates Jakarta EE deployment is being used and that the `@BeforeEach  public void _setup() throws Exception { }` method should return right away.  Otherwise, the `public void setup() throws Exception` should be called.  The EE testing layer will directly call the `public void setup() throws Exception` method which doesn't check for the EE system property.

2. Do the same for each AfterEach method.

The test changes should be done in a way that doesn't change any test logic in already balloted component specifications.

Note that there are 566 references to BeforeEach as per grep -rli org.junit.jupiter.api.BeforeEach --include=*.java| wc -l Note that there are 358 references to AfterEach as per grep -rli org.junit.jupiter.api.AfterEach --include=*.java| wc -l

Pages, Websocket Platform TCK tests does not run in an Test Vehicle. Alwin has raised PR https://github.com/jakartaee/platform-tck/pull/1425 for EL to restore JavaTest classes for Platform TCK.For Persistence, should we restore Javatest classes, as done for JMS?

scottmarlow commented 1 month ago

Pages, Websocket Platform TCK tests does not run in an Test Vehicle. Alwin has raised PR #1425 for EL to restore JavaTest classes for Platform TCK.For Persistence, should we restore Javatest classes, as done for JMS?

As discussed on the Platform TCK call today for EE 11, we should restore the JavaTest classes for Persistence as a new Platform TCK test folder.

The current https://github.com/jakartaee/platform-tck/tree/main/jpa folder source will be dedicated to the (component/standalone) Persistence/JPA TCK.