Slow performance with WildFly Bootable

[blackat]

Hello, we have observed that migrating our application (we made many tests with the same app version) from WildFly 23 to WildFly 23 bootable version there is a degradation in the perfomance of around 30%, the degradation is even more important when the execution run in a Docker container. We have tried to

• migrate to WildFly 26 but we have the same performance gap;
• change the JDK from Amazon Corretto 11 to OpenJDK 11;
• test on Apple M1 and Intel i7 processor;
• have only 1 jar as the hallow jar + the application war.

but still the same performance loss.

Here some questions:

• have you ever remarked such a performance loss?
• are there some parameters we could play with during the build of the image that could have an effect on the performance?
• is there a way we could track the performance loss to understand where it comes from? (except JProfiler or similar)

[jfdenise]

We didn't get such report yet. Did you set a proper amount for Xmx and Xms JVM options? You could add -Xlog:gc*::time JVM option and see if it shows something.

[blackat]

Yes we have set the proper amount of ram, we will try your suggestion

[jfdenise]

Something that you could try is to run the widlfy server from target/bootable-jar-build-artifacts/wildfly/bin/standalone.sh and see if you observe the same loss. The default server JAVA_OPTS are: -server -Xms64m -Xmx512m -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=256m -Djava.net.preferIPv4Stack=true -Djboss.modules.system.pkgs=org.jboss.byteman -Djava.awt.headless=true

[blackat]

Hello, we have tried the following runs

• use wildfly hollow from the target folder
• use wildfly hollow as a jar

it does not make any difference, but what we have noticed is when the hollow jar from a Docker container based on Alpine 3.16 and Amazon Corretto 11 (we use Amazon Corretto outside Docker container as well).

Having a normal WildFly installation in the same Docker base image gives as the same performance of the hollow jar outside the container.

Do you have any idea why the Docker container makes huge performance difference with hollow jar?

[jfdenise]

@blackat , so just to be sure. You have the same issue with a bootable JAR and when running wildfly from the directory target/bootable-build-artifacts/wildfly. Correct? It could be an arch (arm, amd) issue. The image you are building is multi arch?

[blackat]

Hello, yes we have exactly the same issue. Currently we use as base image Alpine 3.16.3 for adm64

$ uname -a
Linux f60872193a6c 5.10.16.3-microsoft-standard-WSL2 #1 SMP Fri Apr 2 22:23:49 UTC 2021 x86_64 Linux

[jfdenise]

Inside a container, you could have container limits. Could you call : java -XshowSettings:system -version inside the container, the output could be of interest? You could try to use "-XX:MaxRAMPercentage=80.0" when starting the bootable JAR, you would then use 80% of the available RAM of the container.

[blackat]

Hello, thanks for you suggestion, here the result of the call

Operating System Metrics:
    Provider: cgroupv1
    Effective CPU Count: 12
    CPU Period: 100000us
    CPU Quota: -1
    CPU Shares: -1
    List of Processors, 12 total:
    0 1 2 3 4 5 6 7 8 9 10 11
    List of Effective Processors, 12 total:
    0 1 2 3 4 5 6 7 8 9 10 11
    List of Memory Nodes, 1 total:
    0
    List of Available Memory Nodes, 1 total:
    0
    Memory Limit: Unlimited
    Memory Soft Limit: Unlimited
    Memory & Swap Limit: Unlimited
    Maximum Processes Limit: Unlimited

openjdk version "11.0.17" 2022-10-18 LTS
OpenJDK Runtime Environment Corretto-11.0.17.8.1 (build 11.0.17+8-LTS)
OpenJDK 64-Bit Server VM Corretto-11.0.17.8.1 (build 11.0.17+8-LTS, mixed mode)

It seems that everything is set to unlimited, so it should use all the available RAM, doesn't it?

[jfdenise]

Yes. What about the gc logs? If nothing shows up you would need to use a perf tool to understand what is going on.

[jfdenise]

@blackat , I am going to close this issue, feel free to re-open it. Thank-you.