search

Karm / mandrel-integration-tests

Integration tests for GraalVM and its Mandrel distribution. Runs Quarkus, Helidon and Micronaut applications and small targeted reproducers. The focus is solely on native-image utility and compilation of Java applications into native executables.
Apache License 2.0
5 stars 3 forks source link

Add integration test for Calendar building. #221

Closed jerboaa closed 7 months ago

jerboaa commented 7 months ago

Related to https://github.com/quarkusio/quarkus/issues/36975

jerboaa commented 7 months ago

Re: CI failures. They all seem caused by the new test (at least for 22.3) which needs: https://github.com/graalvm/mandrel/pull/616

jerboaa commented 7 months ago

LGTM as long as the warning like https://github.com/Karm/mandrel-integration-tests/actions/runs/6850877756/job/18625960493?pr=221#step:7:27515 are going to be fixed by graalvm/mandrel#616 (or whitelisted)?

Good catch. I forgot to git add the pom.xml file. Doh! Should be fixed in the latest push.

Please merge after https://github.com/graalvm/mandrel/pull/616 gets merged and the tests on this PR go green.

OK. Waiting on the CI to complete on that one first.

jerboaa commented 7 months ago

Please merge after graalvm/mandrel#616 gets merged and the tests on this PR go green.

@zakkak On second thought. PR testers use released 22.3.3.1-Final bits, which will not pass this new test. PR testers would have to build from unreleased mandrel/22.3 branch. So I don't think merging of graalvm/mandrel#616 has any influence on these going green. If it matters, 23.0 based PR checks passed. OK to merge?

zakkak commented 7 months ago

You are right. Sure go ahead and merge when ready.

jerboaa commented 7 months ago

I'll see if I can add some conditional logic to only run it on 22.3.5 or so and better.

jerboaa commented 7 months ago

Should now only run for 22.3.5 and better (tested locally):

[INFO] Scanning for projects...
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Build Order:
[INFO] 
[INFO] Native image integration TS                                        [pom]
[INFO] testsuite                                                          [jar]
[INFO] 
[INFO] ----------------< org.graalvm.tests.integration:parent >----------------
[INFO] Building Native image integration TS 1.0.0-SNAPSHOT                [1/2]
[INFO]   from pom.xml
[INFO] --------------------------------[ pom ]---------------------------------
[INFO] 
[INFO] --- clean:3.2.0:clean (default-clean) @ parent ---
[INFO] 
[INFO] --------------< org.graalvm.tests.integration:testsuite >---------------
[INFO] Building testsuite 1.0.0-SNAPSHOT                                  [2/2]
[INFO]   from testsuite/pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- clean:3.2.0:clean (default-clean) @ testsuite ---
[INFO] Deleting /disk/graal/upstream-sources/mandrel-integration-tests/testsuite/target
[INFO] 
[INFO] --- resources:3.3.0:resources (default-resources) @ testsuite ---
[INFO] Copying 1 resource
[INFO] 
[INFO] --- compiler:3.8.1:compile (default-compile) @ testsuite ---
[INFO] No sources to compile
[INFO] 
[INFO] --- build-helper:3.0.0:add-test-source (add-integration-test-source-as-test-sources) @ testsuite ---
[INFO] Test Source directory: /disk/graal/upstream-sources/mandrel-integration-tests/testsuite/src/it/java added.
[INFO] 
[INFO] --- resources:3.3.0:testResources (default-testResources) @ testsuite ---
[INFO] Copying 1 resource
[INFO] 
[INFO] --- compiler:3.8.1:testCompile (default-testCompile) @ testsuite ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 26 source files to /disk/graal/upstream-sources/mandrel-integration-tests/testsuite/target/test-classes
[INFO] /disk/graal/upstream-sources/mandrel-integration-tests/testsuite/src/it/java/org/graalvm/tests/integration/PerfCheckTest.java: /disk/graal/upstream-sources/mandrel-integration-tests/testsuite/src/it/java/org/graalvm/tests/integration/PerfCheckTest.java uses or overrides a deprecated API.
[INFO] /disk/graal/upstream-sources/mandrel-integration-tests/testsuite/src/it/java/org/graalvm/tests/integration/PerfCheckTest.java: Recompile with -Xlint:deprecation for details.
[INFO] 
[INFO] --- surefire:2.22.2:test (default-test) @ testsuite ---
[WARNING] Parameter 'localRepository' is deprecated core expression; Avoid use of ArtifactRepository type. If you need access to local repository, switch to '${repositorySystemSession}' expression and get LRM from it instead.
[INFO] 
[INFO] -------------------------------------------------------
[INFO]  T E S T S
[INFO] -------------------------------------------------------
[INFO] Running org.graalvm.tests.integration.AppReproducersTest
2023-11-14 11:26:04.570 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of QUARKUS_NATIVE_CONTAINER_RUNTIME,quarkus.native.container-runtime as env or sys props, defaulting to docker
2023-11-14 11:26:04.573 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of ROOTLESS_CONTAINER_RUNTIME,rootless.container-runtime as env or sys props, defaulting to false
2023-11-14 11:26:04.574 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of PODMAN_WITH_SUDO,podman.with.sudo as env or sys props, defaulting to true
2023-11-14 11:26:04.574 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of DOCKER_WITH_SUDO,docker.with.sudo as env or sys props, defaulting to false
2023-11-14 11:26:04.574 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of FAIL_ON_PERF_REGRESSION,fail.on.perf.regression as env or sys props, defaulting to true
2023-11-14 11:26:04.576 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of QUARKUS_VERSION_GITSHA,quarkus.version.gitsha as env or sys props, defaulting to nothing (empty string)
2023-11-14 11:26:04.576 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of QUARKUS_NATIVE_BUILDER_IMAGE,quarkus.native.builder-image as env or sys props, defaulting to quay.io/quarkus/ubi-quarkus-mandrel-builder-image:22.3-java17
2023-11-14 11:26:04.576 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of CMD_DEFAULT_TIMEOUT_MS,cmd.default.timeout.ms as env or sys props, defaulting to 10000
2023-11-14 11:26:04.577 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of CMD_LONG_TIMEOUT_MS,cmd.long.timeout.ms as env or sys props, defaulting to 80000
2023-11-14 11:26:04.577 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of GOTO_URL_TIMEOUT_MS,goto.url.timeout.ms as env or sys props, defaulting to 250
2023-11-14 11:26:04.577 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of LONG_GOTO_URL_TIMEOUT_MS,long.goto.url.timeout.ms as env or sys props, defaulting to 60000
2023-11-14 11:26:04.581 INFO  [o.g.t.i.u.v.UsedVersion$MVersion] (runNativeImageVersion) Running command [native-image, --version] to determine Mandrel version used.
2023-11-14 11:26:04.987 INFO  [o.g.t.i.u.v.UsedVersion$MVersion] (of) The test suite runs with Mandrel version 22.3.5-dev installed locally on PATH, JDK 17.0.10.
2023-11-14 11:26:04.998 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of PERFCHECK_TEST_LIGHT_REQUESTS,perfcheck.test.light.requests as env or sys props, defaulting to 100
2023-11-14 11:26:04.999 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of PERFCHECK_TEST_HEAVY_REQUESTS,perfcheck.test.heavy.requests as env or sys props, defaulting to 2
2023-11-14 11:26:04.999 INFO  [o.g.t.i.u.Commands] (getProperty) Failed to detect any of PERFCHECK_TEST_REQUESTS_MX_HEAP_MB,perfcheck.test.requests.mx.heap.mb as env or sys props, defaulting to 2560
2023-11-14 11:26:05.014 INFO  [o.g.t.i.AppReproducersTest] (calendarsBakedIn) Testing app: CALENDARS
2023-11-14 11:26:05.020 INFO  [o.g.t.i.u.Commands$ProcessRunner] (run) Command: [mvn, package]
[INFO] Scanning for projects...
[INFO] 
[INFO] ------------------------< calendars:calendars >-------------------------
[INFO] Building calendars 1
[INFO]   from pom.xml
[INFO] --------------------------------[ jar ]---------------------------------
[INFO] 
[INFO] --- resources:3.3.0:resources (default-resources) @ calendars ---
[INFO] skip non existing resourceDirectory /disk/graal/upstream-sources/mandrel-integration-tests/apps/calendars/src/main/resources
[INFO] 
[INFO] --- compiler:3.10.1:compile (default-compile) @ calendars ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 1 source file to /disk/graal/upstream-sources/mandrel-integration-tests/apps/calendars/target/classes
[INFO] 
[INFO] --- resources:3.3.0:testResources (default-testResources) @ calendars ---
[INFO] skip non existing resourceDirectory /disk/graal/upstream-sources/mandrel-integration-tests/apps/calendars/src/test/resources
[INFO] 
[INFO] --- compiler:3.10.1:testCompile (default-testCompile) @ calendars ---
[INFO] No sources to compile
[INFO] 
[INFO] --- surefire:3.0.0:test (default-test) @ calendars ---
[INFO] No tests to run.
[INFO] 
[INFO] --- jar:3.2.0:jar (default-jar) @ calendars ---
[INFO] Building jar: /disk/graal/upstream-sources/mandrel-integration-tests/apps/calendars/target/calendars.jar
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  1.094 s
[INFO] Finished at: 2023-11-14T11:26:06+01:00
[INFO] ------------------------------------------------------------------------
2023-11-14 11:26:06.958 INFO  [o.g.t.i.u.Commands$ProcessRunner] (run) Command: [native-image, --link-at-build-time=calendar.Main, -jar, target/calendars.jar, target/calendars]
========================================================================================================================
GraalVM Native Image: Generating 'target/calendars' (executable)...
========================================================================================================================
[1/7] Initializing...                                                                                    (2.4s @ 0.16GB)
 Version info: 'GraalVM 22.3.5-dev Java 17 Mandrel Distribution'
 Java version info: '17.0.10-beta+1-202311072331'
 C compiler: gcc (redhat, x86_64, 13.2.1)
 Garbage collector: Serial GC
[2/7] Performing analysis...  [*****]                                                                    (8.2s @ 0.76GB)
   2,832 (73.07%) of  3,876 classes reachable
   3,411 (50.84%) of  6,709 fields reachable
  12,534 (42.90%) of 29,220 methods reachable
      28 classes,     0 fields, and   314 methods registered for reflection
      58 classes,    58 fields, and    52 methods registered for JNI access
       4 native libraries: dl, pthread, rt, z
[3/7] Building universe...                                                                               (1.3s @ 1.27GB)
[4/7] Parsing methods...      [*]                                                                        (0.9s @ 0.45GB)
[5/7] Inlining methods...     [***]                                                                      (0.5s @ 0.85GB)
[6/7] Compiling methods...    [***]                                                                      (5.9s @ 1.85GB)
[7/7] Creating image...                                                                                  (1.9s @ 2.28GB)
   4.04MB (35.85%) for code area:     7,199 compilation units
   6.93MB (61.47%) for image heap:   96,782 objects and 5 resources
 309.55KB ( 2.68%) for other data
  11.27MB in total
------------------------------------------------------------------------------------------------------------------------
Top 10 packages in code area:                               Top 10 object types in image heap:
 689.60KB java.util                                          909.91KB java.lang.String
 328.42KB java.lang                                          893.60KB byte[] for code metadata
 264.68KB java.text                                          837.63KB byte[] for general heap data
 216.40KB java.util.regex                                    622.32KB java.lang.Class
 194.76KB java.util.concurrent                               538.63KB byte[] for java.lang.String
 146.96KB java.math                                          430.55KB java.util.HashMap$Node
 117.08KB java.lang.invoke                                   221.25KB com.oracle.svm.core.hub.DynamicHubCompanion
 114.88KB com.oracle.svm.core.genscavenge                    203.08KB java.util.HashMap$Node[]
 103.84KB java.util.logging                                  158.15KB java.lang.String[]
  95.19KB java.util.stream                                   154.92KB java.util.concurrent.ConcurrentHashMap$Node
   1.77MB for 118 more packages                                1.52MB for 774 more object types
------------------------------------------------------------------------------------------------------------------------
                        0.5s (2.2% of total time) in 18 GCs | Peak RSS: 3.08GB | CPU load: 8.51
------------------------------------------------------------------------------------------------------------------------
Produced artifacts:
 /disk/graal/upstream-sources/mandrel-integration-tests/apps/calendars/target/calendars (executable)
 /disk/graal/upstream-sources/mandrel-integration-tests/apps/calendars/target/calendars.build_artifacts.txt (txt)
========================================================================================================================
Finished generating 'target/calendars' in 22.6s.
2023-11-14 11:26:30.728 INFO  [o.g.t.i.AppReproducersTest] (calendarsBakedIn) Running...
2023-11-14 11:26:30.729 INFO  [o.g.t.i.u.Commands] (runCommand) Command: [./target/calendars]
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 26.466 s - in org.graalvm.tests.integration.AppReproducersTest
[INFO] 
[INFO] Results:
[INFO] 
[INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0
[INFO] 
[INFO] 
[INFO] --- jar:3.3.0:jar (default-jar) @ testsuite ---
[INFO] Building jar: /disk/graal/upstream-sources/mandrel-integration-tests/testsuite/target/testsuite-1.0.0-SNAPSHOT.jar
[INFO] 
[INFO] --- failsafe:2.22.2:integration-test (default) @ testsuite ---
[WARNING] Parameter 'localRepository' is deprecated core expression; Avoid use of ArtifactRepository type. If you need access to local repository, switch to '${repositorySystemSession}' expression and get LRM from it instead.
[INFO] 
[INFO] --- failsafe:2.22.2:verify (default) @ testsuite ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Native image integration TS 1.0.0-SNAPSHOT:
[INFO] 
[INFO] Native image integration TS ........................ SUCCESS [  0.116 s]
[INFO] testsuite .......................................... SUCCESS [ 29.757 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  29.939 s
[INFO] Finished at: 2023-11-14T11:26:31+01:00
[INFO] ------------------------------------------------------------------------

And skipped with a 22.3.2 build:

2023-11-14 11:24:31.837 INFO  [o.g.t.i.u.v.UsedVersion$MVersion] (runNativeImageVersion) Running command [native-image, --version] to determine Mandrel version used.
2023-11-14 11:24:32.309 INFO  [o.g.t.i.u.v.UsedVersion$MVersion] (of) The test suite runs with Mandrel version 22.3.2-final installed locally on PATH, JDK 17.0.7.
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1, Time elapsed: 0.806 s - in org.graalvm.tests.integration.AppReproducersTest
[INFO] 
[INFO] Results:
[INFO] 
[WARNING] Tests run: 1, Failures: 0, Errors: 0, Skipped: 1
[INFO] 
[INFO] 
[INFO] --- jar:3.3.0:jar (default-jar) @ testsuite ---
[INFO] Building jar: /disk/graal/upstream-sources/mandrel-integration-tests/testsuite/target/testsuite-1.0.0-SNAPSHOT.jar
[INFO] 
[INFO] --- failsafe:2.22.2:integration-test (default) @ testsuite ---
[WARNING] Parameter 'localRepository' is deprecated core expression; Avoid use of ArtifactRepository type. If you need access to local repository, switch to '${repositorySystemSession}' expression and get LRM from it instead.
[INFO] 
[INFO] --- failsafe:2.22.2:verify (default) @ testsuite ---
[INFO] ------------------------------------------------------------------------
[INFO] Reactor Summary for Native image integration TS 1.0.0-SNAPSHOT:
[INFO] 
[INFO] Native image integration TS ........................ SUCCESS [  0.120 s]
[INFO] testsuite .......................................... SUCCESS [  4.260 s]
[INFO] ------------------------------------------------------------------------
[INFO] BUILD SUCCESS
[INFO] ------------------------------------------------------------------------
[INFO] Total time:  4.446 s
[INFO] Finished at: 2023-11-14T11:24:32+01:00
[INFO] ------------------------------------------------------------------------