Truffle compilations run in "hosted" mode, i.e. the Truffle runtimes triggers compilations independently of HotSpot's CompileBroker. But the results of Truffle compilations are still stored as ordinary nmethods in HotSpot's code cache (with the help of the JVMCI method jdk.vm.ci.hotspot.HotSpotCodeCacheProvider::installCode()). The regular JIT compilers are controlled by the CompileBroker which is aware of the code cache occupancy. If the code cache runs full, the CompileBroker temporary pauses any subsequent JIT compilations until the code cache gets swept (if running with -XX:+UseCodeCacheFlushing -XX:+MethodFlushing which is the default) or completely shuts down the JIT compilers if running with -XX:+UseCodeCacheFlushing.
Truffle compiled methods can contribute significantly to the overall code cache occupancy and they can trigger JIT compilation stalls if they fill the code cache up. But the Truffle framework itself is neither aware of the current code cache occupancy, nor of the compilation activity of the CompileBroker. If Truffle tries to install a compiled method through JVMCI and the code cache is full, it will silently fail. Currently Truffle interprets such failures as transient errors and basically ignores it. Whenever the corresponding method gets hot again (usually immediately at the next invocation), Truffle will recompile it again just to fail again in the nmethod installation step, if the code cache is still full.
When the code cache is tight, this can lead to situations, where Truffle is unnecessarily and repeatedly compiling methods which can't be installed in the code cache but produce a significant CPU load. Instead, Truffle should poll HotSpot's CompileBroker compilation activity and paus compilations for the time the CompileBroker is pausing JIT compilations (or completely shutdown Truffle compilations if the CompileBroker shut down the JIT compilers).
This PR fixes the problem by checking HotSpot's compilation activity mode in OptimizedTruffleRuntime::submitForCompilation() before actually submitting a compilation task to a compile queue. If the compilation activity mode is RUN_COMPILATION the task is submitted as before without any changes. However, if the compilation activity mode is STOP_COMPILATION (i.e. the CompileBroker has temporarily stopped JIT compilations, we flush the current compile queue (because compiled methods can not be installed anyway) and return null. We also start a timer which can be configured with the new StoppedCompilationRetryDelay parameter (defaults to 1000ms). After StoppedCompilationRetryDelay we submit a new compilation task, even if the compilation activity mode is not RUN_COMPILATION. This can help to trigger a code cache cleanup in situations when there are no JIT compilations, because code cache sweeping is only triggered when new nmethods are installed. Finally, when the compilation activity mode is SHUTDOWN_COMPILATION, we simply shutdown the compilation queue and issue a warning to inform users about the code cache shortage.
I've manually tested the change by running an octane benchmark with a very small code cache. Before the change, we could see the following results:
As you can see, out of 2255 compilations, 1702 have been to no purpose, because they failed in the final installation step because of a full code cache (i.e. BailoutException: Code installation failed: code cache is full: 1702). The other interesting observation is that although the benchmark terminated after 3min11s wall clock time, it actually consumed 11m19s cpu time, because the Truffle compiler threads where continuously doing useless work.
With my changes applied, the picture looks as follows:
RayTrace: 531
----
Score (version 9): 531
[engine] Truffle runtime statistics for engine 1
Compilations : 128
Success : 111
Temporary Bailouts : 17
jdk.vm.ci.code.BailoutException: Code installation failed: code cache is full: 16
org.graalvm.compiler.core.common.CancellationBailoutException: Compilation cancelled.: 1
Permanent Bailouts : 0
Failed : 0
Interrupted : 0
Invalidated : 0
Queues : 410
Dequeues : 283
Compilation temporary disabled due to full code cache.: 277
Target inlined into only caller: 6
Splits : 156
Compilation Accuracy : 1.000000
Queue Accuracy : 0.309756
Compilation Utilization : 0.069104
Remaining Compilation Queue : 0
Time to queue : count= 410, sum=24979066360, min= 88714, average= 60924552.10, max=186708180, maxTarget=blend
Time waiting in queue : count= 128, sum= 22955820, min= 6, average= 179342.35, max= 724639, maxTarget=:anonymous <split-344>
JVMCI CompileBroker Time:
Compile: 0,000 s
Install Code: 0,000 s (installs: 0, CodeBlob total size: 0, CodeBlob code size: 0)
JVMCI Hosted Time:
Install Code: 0,637 s (installs: 124, CodeBlob total size: 1156760, CodeBlob code size: 321496)
nmethod code size : 2460456 bytes
nmethod total size : 4833272 bytes
CodeCache: size=3896Kb used=3464Kb max_used=3537Kb free=431Kb
bounds [0x00007ffff436d000, 0x00007ffff473b000, 0x00007ffff473b000]
total_blobs=1557 nmethods=745 adapters=714
compilation: disabled (not enough contiguous free space left)
stopped_count=5, restarted_count=4
full_count=33
real 3m14,756s
user 3m22,013s
sys 0m1,044s
As you can see, we compile considerably fewer methods and we only have 16 compilation failures due to a full code cache. At the same time we can see that from the 410 methods which have been enqueued for compilation, 277 have been dequeued because of Compilation temporary disabled due to full code cache. Again, the wall clock time of the benchmark was 3m14s but this time, the overall cpu time was just slightly higher with 3m22s because we haven't done such a huge amount of unnecessary compilations any more. Also notice how HotSpot's code cache full_count, which is incremented every time when a nmethod can't be installed because of a full code cache, dropped from 3200 to 33.
These example were run with -XX:+UseCodeCacheFlushing -XX:+MethodFlushing. If the JVM is configured with -XX:-UseCodeCacheFlushing, the benefits of this change are even higher.
Truffle compilations run in "hosted" mode, i.e. the Truffle runtimes triggers compilations independently of HotSpot's
CompileBroker. But the results of Truffle compilations are still stored as ordinary nmethods in HotSpot's code cache (with the help of the JVMCI methodjdk.vm.ci.hotspot.HotSpotCodeCacheProvider::installCode()). The regular JIT compilers are controlled by theCompileBrokerwhich is aware of the code cache occupancy. If the code cache runs full, theCompileBrokertemporary pauses any subsequent JIT compilations until the code cache gets swept (if running with-XX:+UseCodeCacheFlushing -XX:+MethodFlushingwhich is the default) or completely shuts down the JIT compilers if running with-XX:+UseCodeCacheFlushing.Truffle compiled methods can contribute significantly to the overall code cache occupancy and they can trigger JIT compilation stalls if they fill the code cache up. But the Truffle framework itself is neither aware of the current code cache occupancy, nor of the compilation activity of the
CompileBroker. If Truffle tries to install a compiled method through JVMCI and the code cache is full, it will silently fail. Currently Truffle interprets such failures as transient errors and basically ignores it. Whenever the corresponding method gets hot again (usually immediately at the next invocation), Truffle will recompile it again just to fail again in the nmethod installation step, if the code cache is still full.When the code cache is tight, this can lead to situations, where Truffle is unnecessarily and repeatedly compiling methods which can't be installed in the code cache but produce a significant CPU load. Instead, Truffle should poll HotSpot's
CompileBrokercompilation activity and paus compilations for the time theCompileBrokeris pausing JIT compilations (or completely shutdown Truffle compilations if theCompileBrokershut down the JIT compilers).The corresponding JVMCI change is tracked under JDK-8344727: [JVMCI] Export the CompileBroker compilation activity mode for Truffle compiler control.
This PR fixes the problem by checking HotSpot's compilation activity mode in
OptimizedTruffleRuntime::submitForCompilation()before actually submitting a compilation task to a compile queue. If the compilation activity mode isRUN_COMPILATIONthe task is submitted as before without any changes. However, if the compilation activity mode isSTOP_COMPILATION(i.e. theCompileBrokerhas temporarily stopped JIT compilations, we flush the current compile queue (because compiled methods can not be installed anyway) and returnnull. We also start a timer which can be configured with the newStoppedCompilationRetryDelayparameter (defaults to 1000ms). AfterStoppedCompilationRetryDelaywe submit a new compilation task, even if the compilation activity mode is notRUN_COMPILATION. This can help to trigger a code cache cleanup in situations when there are no JIT compilations, because code cache sweeping is only triggered when new nmethods are installed. Finally, when the compilation activity mode isSHUTDOWN_COMPILATION, we simply shutdown the compilation queue and issue a warning to inform users about the code cache shortage.I've manually tested the change by running an octane benchmark with a very small code cache. Before the change, we could see the following results:
As you can see, out of 2255 compilations, 1702 have been to no purpose, because they failed in the final installation step because of a full code cache (i.e.
BailoutException: Code installation failed: code cache is full: 1702). The other interesting observation is that although the benchmark terminated after 3min11s wall clock time, it actually consumed 11m19s cpu time, because the Truffle compiler threads where continuously doing useless work.With my changes applied, the picture looks as follows:
As you can see, we compile considerably fewer methods and we only have 16 compilation failures due to a full code cache. At the same time we can see that from the 410 methods which have been enqueued for compilation, 277 have been dequeued because of
Compilation temporary disabled due to full code cache. Again, the wall clock time of the benchmark was 3m14s but this time, the overall cpu time was just slightly higher with 3m22s because we haven't done such a huge amount of unnecessary compilations any more. Also notice how HotSpot's code cachefull_count, which is incremented every time when a nmethod can't be installed because of a full code cache, dropped from 3200 to 33.These example were run with
-XX:+UseCodeCacheFlushing -XX:+MethodFlushing. If the JVM is configured with-XX:-UseCodeCacheFlushing, the benefits of this change are even higher.Fixes #10133