Document DirectByteBuffer memory limits

[pdbain-ibm]

Subject: Direct Byte Buffer memory limit

Suggestion: In re https://github.com/eclipse/openj9/issues/2604, (https://www.ibm.com/support/knowledgecenter/en/SSYKE2_8.0.0/com.ibm.java.vm.80.doc/docs/xxmaxdirectmemorysize.html) refers to the classic J9 implementation of direct byte buffers. Please note that OpenJ9 uses the OpenJDK implementation. The limit on memory used for direct byte buffers is now based on maximum heap size (currently about 85%), but can be changed using the -XX:MaxDirectMemorySize= option.

[SueChaplain]

Hi @pdbain-ibm,

I'm trying to understand this. It sounds like OpenJ9 now allocates the limit differently, which will be reflected in any binaries that embed OpenJ9, including OpenJDK and the IBM SDK for Java 8. Is this correct? So a change in behaviour that requires us to update the following sentence in https://www.eclipse.org/openj9/docs/xxmaxdirectmemorysize/

By default, the VM does not set a limit on how much memory is reserved for Direct Byte Buffers. A soft limit of 64 MB is set, which the VM automatically expands in 32 MB chunks, as required.

This change would also need to be reflected in the SDK docs? Is this change included for OpenJ9 0.10.0 please?

[pdbain-ibm]

@SueChaplain

OpenJ9 now allocates the limit differently, which will be reflected in any binaries that embed OpenJ9

That is correct.

including OpenJDK and the IBM SDK for Java 8.

The IBM SDK for Java 8 uses a different version of the Java class library and maintains the legacy behaviour. OpenJDK with OpenJ9 uses the new behaviour.

@pshipton FYI.

[SueChaplain]

@pd-bain Sorry, another question relating to Java core. Some information used to be recorded in the NATIVEMEMINFO section related to DBB. Is this still recorded with the new behaviour please?

[doveye]

@pd-bain Hi Peter. I plan on changing the following sentence in the OpenJ9 topic at https://www.eclipse.org/openj9/docs/xxmaxdirectmemorysize/ ...

By default, the JVM does not set a limit on how much memory is reserved for Direct Byte Buffers. A soft limit of 64 MB is set, which the JVM automatically expands in 32 MB chunks, as required.

to this:

By default, the JVM sets a limit on how much memory is reserved for Direct Byte Buffers. This limit is approximately 85% of the maximum heap size. Use this option to change the limit if required.

We will retain the original information in the IBM SDK docs. I can't see any other topic that needs changing (unless the NATIVEMEMINFO content that Sue mentioned, which is at https://www.ibm.com/support/knowledgecenter/SSYKE2_8.0.0/com.ibm.java.vm.80.doc/docs/javadump_tags_nativememinfo.html, needs changing too).

Can you confirm whether this is ok please, thanks.

[pdbain-ibm]

@SueChaplain NATIVEMEMINFO shows memory actually requested. As this change affects limits, it should not change the NATIVEMEMINFO section. I also don't see any specific mention of DBB there anyway.

@doveye Since we don't actually "reserve" memory for DBBs, I would reword this:

By default, the JVM sets a limit on how much memory is reserved for Direct Byte Buffers. This limit is approximately 85% of the maximum heap size.

to

The JVM limits heap memory used for Direct Byte Buffers to approximately 85% of the maximum heap size by default.

@pshipton FYI.

Thanks -p

[doveye]

OpenJ9 docs are now updated: https://www.eclipse.org/openj9/docs/xxmaxdirectmemorysize/

[pshipton]

Note the DBB line in the javacore is still there and working in OpenJ9 5MEMUSER | | | +--Direct Byte Buffers: 528 bytes / 6 allocations

[doveye]

Thanks Peter :)

[gjdeval]

The JVM limits heap memory used for Direct Byte Buffers to approximately 85% of the maximum heap size by default.

"heap memory" is incorrect - DBBs are stored in native memory, though a small reference object to access them is located on the heap ... but the limit is on the native memory consumption, not the heap memory used

[kgibm]

@doveye

We will retain the original information in the IBM SDK docs

I believe the IBM SDK still retains the old behavior but I can't find it documented anywhere

[doveye]

@kgibm The old behavoiur was documented up until March but it appears to have been erroneously overwritten, so thanks for spotting it! The SDK topic for -XX:MaxDirectMemorySize should say "By default, the JVM does not set a limit on how much memory is used for Direct Byte Buffers. A soft limit of 64 MB is set, which the JVM automatically expands in 32 MB chunks, as required." instead of "By default, the VM limits the amount of heap memory used for Direct Byte Buffers to 87.5% of the maximum heap size." This will be fixed.

There is a section for "Modifications to OpenJ9 behavior" in the SDK docs, which does still document the difference.

@gjdeval If there's content that needs changing in the OpenJ9 docs, please open a new issue in this repo rather than commenting on an old, closed one (you can link to the old issue for reference), otherwise the requirement could get lost. Thanks.

[gjdeval]

@doveye Opened https://github.com/eclipse-openj9/openj9-docs/issues/947 for the problem noted above.