Integration tests for GraalVM and its Mandrel distribution. Runs Quarkus, Helidon and Micronaut applications and small targeted reproducers. The focus is solely on native-image utility and compilation of Java applications into native executables.
Apache License 2.0
5
stars
3
forks
source link
Make perf threshold fail on a certain percentage and not hard comparison <= #59
Test suite reports how big is the difference between a value and its threshold when it fails:
org.opentest4j.AssertionFailedError:
Application JFR_SMOKE_BUILDER_IMAGE in mode jvm_jfr took 7095 ms to finish,
which is over 6924 ms threshold by 2%.
==> expected: <true> but was: <false>
The decision whether to fail a test should be made based on the percentage, i.e. 2% should not fail the test, rather than on a simple comparison of value to its threshold.
Test suite reports how big is the difference between a value and its threshold when it fails:
The decision whether to fail a test should be made based on the percentage, i.e. 2% should not fail the test, rather than on a simple comparison of value to its threshold.