To accept OpenJDK option +UseG1GC to enable startup of elasticsearch application

[bhavanisn]

Description: Investigating to run elasticsearch with Semeru runtimes reveals that few OpenJDK options is essential to use the application. The initial results and experiments are on ppc64le, but it should not differ for other platforms running Semeru.

Elasticsearch is very tightly coupled with OpenJDK and it expects all distributions to be compatible with it. So, by default ES thinks that all the OpenJDK options are supported and somehow the option UseG1GC is mandatory for running elasticsearch. If the JVM does not implement this option, then ES will throw an exception during its startup stage (previous versions gives an error while processing JVM options).

es02_1    | Exception in thread "main" java.lang.IllegalStateException: JVM option [UseG1GC] was unexpectedly missing. Elasticsearch requires this option to be present.          
es02_1    |     at org.elasticsearch.server.cli.JvmErgonomics.getRequiredOption(JvmErgonomics.java:98)                                                                                    
es02_1    |     at org.elasticsearch.server.cli.JvmErgonomics.usingG1GcWithoutCommandLineOriginOption(JvmErgonomics.java:91)                                                              
es02_1    |     at org.elasticsearch.server.cli.JvmErgonomics.tuneG1GCHeapRegion(JvmErgonomics.java:67)                                                                                   
es02_1    |     at org.elasticsearch.server.cli.JvmErgonomics.choose(JvmErgonomics.java:44)                                                                                               
es02_1    |     at org.elasticsearch.server.cli.JvmOptionsParser.jvmOptions(JvmOptionsParser.java:139)                                                                                    
es02_1    |     at org.elasticsearch.server.cli.JvmOptionsParser.determineJvmOptions(JvmOptionsParser.java:91)                                                                            
es02_1    |     at org.elasticsearch.server.cli.ServerProcess.createProcess(ServerProcess.java:208)                                                                                       
es02_1    |     at org.elasticsearch.server.cli.ServerProcess.start(ServerProcess.java:104)                                                                                               
es02_1    |     at org.elasticsearch.server.cli.ServerProcess.start(ServerProcess.java:88)                                                                                                
es02_1    |     at org.elasticsearch.server.cli.ServerCli.startServer(ServerCli.java:239)                                                                                                 
es02_1    |     at org.elasticsearch.server.cli.ServerCli.execute(ServerCli.java:100)                                                                                                     
es02_1    |     at org.elasticsearch.common.cli.EnvironmentAwareCommand.execute(EnvironmentAwareCommand.java:54)                                                                          
es02_1    |     at org.elasticsearch.cli.Command.mainWithoutErrorHandling(Command.java:85)                                                                                                
es02_1    |     at org.elasticsearch.cli.Command.main(Command.java:50)                                                                                                                    
es02_1    |     at org.elasticsearch.launcher.CliToolLauncher.main(CliToolLauncher.java:64)          

Previous efforts from Azul Prime JDK team to add a fix where a error was printed while processing JVM options were on vain. In this PR from the Azul PrimeJDK’s effort to push a fix to elasticsearch which ignores check to this option and be able to startup without UseG1GC option specified. This PR was rejected by ES community so as to making their product to be compatible with OpenJDK standards. As a response to this request, they added the code where an exception is raised in early stages of startup of ES if this option is not supported by JVM.

https://github.com/elastic/elasticsearch/pull/93197

Some useful information on UseG1GC option requirement: ES uses UseG1GC option to tune these heap options: -XX:G1HeapRegionSize -XX:InitiatingHeapOccupancyPercent -XX:G1ReservePercent

ES verifies if all the above mentioned vm options are supported by JVM by running java command with -XX:+PrintFlagsFinal before actually bootstrapping the JVM:

java / these options are from ES jvm config file / -XX:+UseG1GC -Djava.io.tmpdir=/tmp/elasticsearch-14347269605287194680 -XX:+HeapDumpOnOutOfMemoryError -XX:+ExitOnOutOfMemoryError -XX:HeapDumpPath=data -XX:ErrorFile=logs/hs_err_pid%p.log -Xlog:gc,gc+age=trace,safepoint:file=logs/gc.log:utctime,level,pid,tags:filecount=32,filesize=64m / Options from ES jvm config file END */ -XX:+PrintFlagsFinal -version

This brings us to the option to be able to show that the options -XX:+UseG1GC -XX:G1HeapRegionSize -XX:InitiatingHeapOccupancyPercent -XX:G1ReservePercent are accepted in Semeru to be able to run ES on our systems.

1. One easy way is to just accept options and ignore it which is also quick to deliver.
2. Other is to support the option which involves longer development cycle.

FYI.. Couple of more issues were uncovered during this investigation along with: First, there is no Power/Z platform enablement for ES. We need to apply a patch to be able to build ES on Power platform. Planning to open a PR to be able to add ppc64le platform support. Second, the current vendor support is only "adoptium", "openjdk", "zulu”. For Semeru Java we need to add ibm as a vendor. could be done only when semeru can support G1GC option.

The changes required for G1GC is independent of platform. If required to build for ppc64le build, apply this patch to be able to build on Power platform with OpenJDK 17.x.x version is below.

es.ppc64le.patch

This patch should enable ES to successfully build and run out of box with OpenJDK/Hotspot on Power. For Semeru additional changes are required.

[bhavanisn]

@zl-wang @patilca

[zl-wang]

those options are currently "accepted and ignored" by Semeru already. your ask is more in the direction of faking/simulating those option processing to the extent that ES code "thinks" they are supported.

[bhavanisn]

@zl-wang yes that is correct. Basically that should be part of supported global flags printed as part of the option XX:+PrintFlagsFinal. Hoping that ES will not change the way of verifying supported options.

[pshipton]

@dmitripivkine @TobiAjila fyi

[tajila]

The next step is to add a compatibilty mode that enables openj9 to run elasticsearch workloads, https://github.com/eclipse-openj9/openj9/issues/18265

[tajila]

This can be closed now