search

Karm / mandrel-integration-tests

Integration tests for GraalVM and its Mandrel distribution. Runs Quarkus, Helidon and Micronaut applications and small targeted reproducers. The focus is solely on native-image utility and compilation of Java applications into native executables.
Apache License 2.0
5 stars 3 forks source link

Investigate HELIDON_QUICKSTART_SE increased RSS in GHAs. #226

Open jerboaa opened 7 months ago

jerboaa commented 7 months ago

We apparently see a ~2MB increase in RSS size in GHA:

 Error:    RuntimesSmokeTest.helidonQuickStart:228->testRuntime:77->testRuntime:163 
Application HELIDON_QUICKSTART_SE in mode none consumed 78764 kB of RSS memory , which is over 78368 kB threshold by 1%.
 ==> expected: <true> but was: <false>

The reference value for 23.1 is: 76647 Kb This needs to be investigated.

See: https://github.com/Karm/mandrel-integration-tests/issues/198#issuecomment-1825698339

jerboaa commented 7 months ago

CI failure is here: https://github.com/graalvm/mandrel/actions/runs/6968236452/job/18962635519#step:11:12851

jerboaa commented 7 months ago

There has been a threshold bump in https://github.com/Karm/mandrel-integration-tests/pull/227 but we need to do the threshold handling per quarkus version probably.

Karm commented 7 months ago

@jerboaa Helidon has nothing to do with Quarkus. Our Helidon version is very old, the last time we updated Helidon was in April 2021: https://github.com/Karm/mandrel-integration-tests/commit/9cab8f92186e975d8f807a56e7844fcb893180fe

If the RSS is off, it's either:

jerboaa commented 7 months ago

Which confirms it needs some investigation?

Karm commented 7 months ago

Which confirms it needs some investigation?

Indeed. A rare occasion where it seems we have fewer moving parts as the app version is the same.

zakkak commented 7 months ago

Running locally I get the following numbers which indicate no significant increase between 23.1, 23.1.1 and graal/master on Linux (~I will look into windows next~ apparently the helidon test is not ran on Windows).

Mandrel version RSS after startup RSS after 4 requests
Mandrel graal/master with JDK 22+25 41472 61184
Mandrel-23.1.1.1-deve19e4e528fd 41472 61440
Mandrel-23.1.0.0-Final 41728 61952
Mandrel-23.0.2.1-Final 40704 59904
Mandrel-22.3.4.0-Final 39936 58624

The 4 requests are:

zakkak commented 7 months ago

Interestingly, the only time it passed was in https://github.com/zakkak/mandrel/actions/runs/6944009708/job/18891435883 (while testing before merging the PR that enabled these tests) and according to the logs it used github runner 2.311.0 which is the same runner used in the failing https://github.com/graalvm/mandrel/actions/runs/6968236452/job/18962635519#step:11:12851 ...

jerboaa commented 7 months ago

Can we disable this test again - until we can reliably enable it again - as it's causing a lot of noise?

zakkak commented 7 months ago

Can we disable this test again - until we can reliably enable it again - as it's causing a lot of noise?

Done in https://github.com/graalvm/mandrel/pull/626

jerboaa commented 7 months ago

Thanks!

zakkak commented 7 months ago

Interestingly, the only time it passed was in https://github.com/zakkak/mandrel/actions/runs/6944009708/job/18891435883 (while testing before merging the PR that enabled these tests)

Turns out that's not true.

The test was passing in https://github.com/graalvm/mandrel/actions/runs/6999473951/job/19039243355#step:11:13026

So the first known bad commit seems to be 61edae46e0287ecaee7f5b3ef728ab4a6d465c02 in https://github.com/graalvm/mandrel/actions/runs/7026828114/job/19120880487

The last known working commit seems to be 670b43cac8587d27d7810af41c22ca3cf161df23 in https://github.com/graalvm/mandrel/actions/runs/7012992753/job/19078995046