Document option -XX:[+|-]JITServerAOTCacheIgnoreLocalSCC

[cjjdespres]

Issue or pull request number:

Introduced in: https://github.com/eclipse-openj9/openj9/pull/18301 Made the default in: https://github.com/eclipse-openj9/openj9/pull/19585

Overview:

This was previously an undocumented experimental option in 0.44.0, but since it's now enabled by default in 0.46.0, it should be documented.

The option -XX:[+|-]JITServerAOTCacheIgnoreLocalSCC only has an effect when specified at a client of a JITServer, when that client is using the JITServer AOT cache mechanism. If enabled, the client will ignore its local shared classes cache when JITServer AOT cache compilations are performed. The client will still load classes and methods from its local SCC as appropriate; it will simply not otherwise look up or store information in the local SCC during JITServer AOT cache compilations. This allows the client to use the JITServer's AOT cache without having a suitable local SCC set up, but also means that the client will store few, if any, new methods in its local SCC.

If disabled, the client will look up and store data, including received cached methods, in its local SCC during JITServer AOT cache compilations. Clients disabling this option must have a local SCC set up in order to use the JITServer AOT cache.

Release target:

OpenJ9 0.46.0

Applies to the following JDK versions:

All supported JDKs

Applies to the following platforms:

All platforms supporting JITServer (Linux X|P|Z).

For new command line options:

• -XX:[+|-]JITServerAOTCacheIgnoreLocalSCC
• Default value: On

[cjjdespres]

Attn @mpirvu.

[Sreekala-Gopakumar]

@cjjdespres - Could you please check the following content and let me know if the information is correct and whether it needs changes.

-XX:[+|-]JITServerAOTCacheIgnoreLocalSCC

This option enables or disables storing and loading of JITServer AOT cache without a local shared classes cache.

Syntax

    -XX:[+|-]JITServerAOTCacheIgnoreLocalSCC

Setting	Effect	Default
-XX:+JITServerAOTCacheIgnoreLocalSCC	Enable	:fontawesome-solid-check:{: .yes aria-hidden="true"}yes
-XX:-JITServerAOTCacheIgnoreLocalSCC	Disable

Explanation

The JITServer AOT cache feature requires the client VM to use a shared class cache with some empty space and write permissions. The AOT code that is fetched from the server is first stored in the cache and then loaded from there, relocated and stored in the code cache. However, dependency on the local cache has some usability constraints when the cache is embedded in a container. Embedding the cache in a container to speed up the applications start-up is typically done in layers, with one shared classes cache layer per container image layer. Such cache layers are trimmed to just the right size and marked as read-only. Local AOT compilations are disabled when the local shared classes cache is read-only. The server AOT cache is only useful if it can store methods not present in the read-only local shared classes cache. For the server to run the AOT compilation of these new methods, the clients must store new ROM classes and class chains in their local shared classes cache, which is impossible if the cache is read-only.

With the -XX:+JITServerAOTCacheIgnoreLocalSCC option, the client ignores its local shared classes cache at the time of JITServer AOT cache compilations. The client still loads classes and methods from its local cache but does not look up or store information in the local cache during JITServer AOT cache compilations. The client can use the JITServer server's AOT cache without having a suitable local cache set up and also the client need to store only a few, if any, new methods in its local cache. Thus, you can improve performance.

You can disable this feature with the -XX:-JITServerAOTCacheIgnoreLocalSCC option. If disabled, the client looks up and stores data, including the received cached methods, in its local shared classes cache during the JITServer AOT cache compilations. Clients must have a local shared classes cache set up in order to use the JITServer AOT cache.

The option -XX:[+|-]JITServerAOTCacheIgnoreLocalSCC is applicable only when specified at a client of a JITServer server and when that client uses the JITServer AOT cache mechanism.

[cjjdespres]

The JITServer AOT cache feature requires the client VM to use a shared class cache with some empty space and write permissions.

I don't think we need to lead with this - since this option is the default, it's not actually true that the JITServer AOT cache requires the client to have a suitable local SCC. It's just that this option allows the user to switch back to the old behaviour of not ignoring the local SCC, which requires a writable SCC with enough free space. Also, this option is performance neutral at the moment - its main benefit is that it makes the JITServer AOT cache feature easier to use by removing the need for a local SCC.

Could the description be simpler? Something like,

This option, used at JITServer client VMs, controls how the JITServer AOT cache feature interacts with the local shared classes cache.

The default behaviour of the client when using the JITServer AOT cache is to bypass its local shared classes cache (if one is set up) during JITServer AOT cache compilations. Class sharing and local AOT method loading will still occur if possible, but no additional data, such as class data or methods received from the server, will be stored in or loaded from the local SCC during these compilations. This behaviour allows the JITServer AOT cache to be used at the client whether or not a local SCC is present.

If you disable this option, the client will not bypass the local SCC during JITServer AOT cache compilations. The client will instead store new data, including methods received from the server's AOT cache, in the local SCC prior to using them. Clients with this option disabled must have a local SCC set up, with write permissions and some free space, in order to use the JITServer AOT cache.

Maybe @mpirvu has some input on what exactly we'd like to include in the documentation here.

[Sreekala-Gopakumar]

@mpirvu - Could you please let me know what needs to be included. Thanks!

[mpirvu]

I like Christian's simplified version better. A few tweaks:

---

This option, used at JITServer client VMs, controls how the JITServer AOT cache feature interacts with the local shared classes cache (SCC) at the client.

Starting with OpenJ9 Release 0.46.0, the default behaviour of the client when using the JITServer AOT cache is to bypass its local shared classes cache (if one is set up) during JITServer AOT cache compilations. Class sharing and local AOT method loading will still occur if possible, but no additional data, such as class data or methods received from the server, will be stored in the local SCC during these compilations. This behaviour allows the clients to take advantage of the JITServer AOT cache irrespective of the presence of a local SCC.

If you disable this option, the client will not bypass the local SCC during JITServer AOT cache compilations. The client will instead store new data, including methods received from the server's AOT cache, in the local SCC prior to using them. Clients with this option disabled must have a local SCC set up, with write permissions and some free space, in order to use the JITServer AOT cache.