Please note, this Feature Test Summary provided is used to claim Java 17 support for Open Liberty and not to introduce any new or modified Open Liberty functionality.
Normally FAT testing is designed to validate the functionality of a new or changed feature in Open Liberty. Since there is no new OL functionality to test, the associated FAT test is just a sanity check to make sure we are running on Java 17. To accomplish that, this FAT uses a WAR file for testing that has been compiled at Java 17 using functionality that is specific to Java 17.
The real testing validation for Java 17 is done via our entire suite of Open Liberty and WebSphere Liberty FAT buckets.
What functionality is new or modified by this feature?
No new Liberty functionality is added or changed by this feature
What are the positive and negative tests for that functionality?
This FAT adds one simple test to ensure we are running on Java 17 by using functionality that is specific to Java 17. It uses an Enhanced Pseudo-Random Number Generator and a new algorithm (L64X128MixRandom), both are part of JEP 356, to randomly produce either a String, integer, long, boolean, float or double value. Then that value is displayed using Patterning Matching for switch, which is a preview feature in Java 17 and part of JEP 406.
In this case, there is no correlation between what is being performed, random number generation, to what is actually being tested, which is verification that we are running on Java 17.
Testing specifics
For several weeks now, we have been running nightly (M-F) Java 17 builds using the latest Java 17 OpenJDK Hotspot builds and tracking defects. This build runs in lite mode against all the Open and WebSphere Liberty FATs. We also periodically run a build against all the same FATs in full mode to make sure we discover any Java 17 specific defects.
Confidence Level
4 - We have delivered all automated testing we believe is needed for the golden paths of this feature and have good coverage of the error/outlying scenarios. While more testing of the error/outlying scenarios could be added we believe there is minimal risk here and the cost of providing these is considered higher than the benefit they would provide.
ayoho
commented
3 years ago
Test Strategy
Please note, this Feature Test Summary provided is used to claim Java 17 support for Open Liberty and not to introduce any new or modified Open Liberty functionality.
Normally FAT testing is designed to validate the functionality of a new or changed feature in Open Liberty. Since there is no new OL functionality to test, the associated FAT test is just a sanity check to make sure we are running on Java 17. To accomplish that, this FAT uses a WAR file for testing that has been compiled at Java 17 using functionality that is specific to Java 17.
The real testing validation for Java 17 is done via our entire suite of Open Liberty and WebSphere Liberty FAT buckets.
List of FAT projects affected
io.openliberty.java.internal_fat- https://github.com/OpenLiberty/open-liberty/pull/18509Test strategy
Enhanced Pseudo-Random Number Generatorand a new algorithm (L64X128MixRandom), both are part of JEP 356, to randomly produce either a String, integer, long, boolean, float or double value. Then that value is displayed usingPatterning Matching for switch, which is a preview feature in Java 17 and part of JEP 406.Confidence Level
4 - We have delivered all automated testing we believe is needed for the golden paths of this feature and have good coverage of the error/outlying scenarios. While more testing of the error/outlying scenarios could be added we believe there is minimal risk here and the cost of providing these is considered higher than the benefit they would provide.