Closed fniephaus closed 7 months ago
Hello Fabio,
thank you for reporting this. We addressed this class of problems in this commit: https://github.com/oracle/fastr/commit/685fc85ea4d68b231a31761776e56b2c8a7de185. It would be great if you could try reproducing with the latest development version of FastR.
/CC @chumer
Thanks for the info, @steve-s. I've had a look at the docs, but couldn't find instructions for building an updated GraalVM component. Could you provide a pointer? And am I right in assuming that the updated GraalVM component will run on 19.2.0 without a problem?
And am I right in assuming that the updated GraalVM component will run on 19.2.0 without a problem? You can try, but i wouldn't assume it works.
In general the GraalVM and installable components are built from the vm
directory in this repository.
Since the R installable doesn't have any native images i would do that:
$ env FASTR_RELEASE=true mx --dynamicimports fastr build
$ env FASTR_RELEASE=true mx --dynamicimports fastr paths R_INSTALLABLE
The second command will print the path to the installable file.
You could also build a GraalVM with R and js with:
env FASTR_RELEASE=true mx --dynamicimports fastr,/graal-js,/substratevm build
If you only use the --jvm
mode, i would remove /substratevm
from the list to make it build faster.
Note: you can save yourself some build time if you also export FASTR_NO_RECOMMENDED=true
so that FastR doesn't download and install the recommended R packages (provided that you don't need them).
Thanks for the help, it looks like I managed to obtain a new r-installable.jar
and install all my dependencies on 19.2.0. I'll update this issue after the server has been running for a while...
This one didn't take long:
[truffle] opt fail ggproto |Reason jdk.vm.ci.code.BailoutException: Code installation failed: code is too large
jdk.vm.ci.code.BailoutException: Code installation failed: code is too large
at jdk.vm.ci.hotspot.HotSpotCodeCacheProvider.installCode(HotSpotCodeCacheProvider.java:150)
at org.graalvm.compiler.core.target.Backend.createInstalledCode(Backend.java:194)
at org.graalvm.compiler.core.target.Backend.createInstalledCode(Backend.java:147)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:534)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:455)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:637)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:579)
at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:178)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:280)
at org.graalvm.compiler.truffle.compiler.hotspot.libgraal.HotSpotToSVMEntryPoints.doCompile(HotSpotToSVMEntryPoints.java:253)
These two are new:
[truffle] opt fail gList |Reason org.graalvm.compiler.code.SourceStackTraceBailoutException$1: Object of type Lorg/graalvm/compiler/truffle/runtime/FrameWithoutBoxing; should not be materialized (must not let virtual object escape at node 874|EndNode):
org.graalvm.compiler.code.SourceStackTraceBailoutException$1: Object of type Lorg/graalvm/compiler/truffle/runtime/FrameWithoutBoxing; should not be materialized (must not let virtual object escape at node 874|EndNode):
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callRoot(OptimizedCallTarget.java)
Caused by: org.graalvm.compiler.graph.VerificationError: Object of type Lorg/graalvm/compiler/truffle/runtime/FrameWithoutBoxing; should not be materialized (must not let virtual object escape at node 874|EndNode):
at org.graalvm.compiler.nodes.virtual.EnsureVirtualizedNode.ensureVirtualFailure(EnsureVirtualizedNode.java:98)
at org.graalvm.compiler.nodes.virtual.CommitAllocationNode.lower(CommitAllocationNode.java:117)
at org.graalvm.compiler.phases.common.LoweringPhase$Round.process(LoweringPhase.java:475)
at org.graalvm.compiler.phases.common.LoweringPhase$Round.access$200(LoweringPhase.java:351)
at org.graalvm.compiler.phases.common.LoweringPhase$Round$ProcessFrame.preprocess(LoweringPhase.java:415)
at org.graalvm.compiler.phases.common.LoweringPhase.processBlock(LoweringPhase.java:574)
at org.graalvm.compiler.phases.common.LoweringPhase$Round.run(LoweringPhase.java:400)
at org.graalvm.compiler.phases.Phase.run(Phase.java:49)
at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:139)
at org.graalvm.compiler.phases.PhaseSuite.run(PhaseSuite.java:209)
at org.graalvm.compiler.phases.common.IncrementalCanonicalizerPhase.run(IncrementalCanonicalizerPhase.java:56)
at org.graalvm.compiler.phases.common.IncrementalCanonicalizerPhase.run(IncrementalCanonicalizerPhase.java:38)
at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:139)
at org.graalvm.compiler.phases.common.LoweringPhase.lower(LoweringPhase.java:271)
at org.graalvm.compiler.phases.common.LoweringPhase.run(LoweringPhase.java:264)
at org.graalvm.compiler.phases.common.LoweringPhase.run(LoweringPhase.java:94)
at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:139)
at org.graalvm.compiler.phases.PhaseSuite.run(PhaseSuite.java:209)
at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:139)
at org.graalvm.compiler.core.GraalCompiler.emitFrontEnd(GraalCompiler.java:220)
at org.graalvm.compiler.core.GraalCompiler.compile(GraalCompiler.java:144)
at org.graalvm.compiler.core.GraalCompiler.compileGraph(GraalCompiler.java:129)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:521)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:455)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:637)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:579)
at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:178)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:280)
at org.graalvm.compiler.truffle.compiler.hotspot.libgraal.HotSpotToSVMEntryPoints.doCompile(HotSpotToSVMEntryPoints.java:253)
[truffle] opt fail @16f322e |Reason jdk.vm.ci.code.BailoutException: Code installation failed: code is too large
jdk.vm.ci.code.BailoutException: Code installation failed: code is too large
at jdk.vm.ci.hotspot.HotSpotCodeCacheProvider.installCode(HotSpotCodeCacheProvider.java:150)
at org.graalvm.compiler.core.target.Backend.createInstalledCode(Backend.java:194)
at org.graalvm.compiler.core.target.Backend.createInstalledCode(Backend.java:147)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:534)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:455)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:637)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:579)
at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:178)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:280)
at org.graalvm.compiler.truffle.compiler.hotspot.libgraal.HotSpotToSVMEntryPoints.doCompile(HotSpotToSVMEntryPoints.java:253)
Could you re-try with --R.PromiseCacheSize=0
and -Dgraal.TruffleFunctionInlining=false
. It seems that our previous approach worked better even though not fully (just one failed compilation).
Running again with the two flags you suggested, @steve-s. Over night, these two errors showed up:
[truffle] opt fail rep.unit |Reason org.graalvm.compiler.code.SourceStackTraceBailoutException$1: Frame escapes at: 21788|MethodCallTarget#HotSpotMethod<InlineCacheNode.evalPromise(Frame, Closure)>
properties:{invokeKind=Static, returnStamp=a Ljava/lang/Object;, targetMethod=HotSpotMethod<InlineCacheNode.evalPromise(Frame, Closure)>, profile=null, stamp=void}
arguments: [13766|NewFrame, 21786|LoadField#closure]
org.graalvm.compiler.code.SourceStackTraceBailoutException$1: Frame escapes at: 21788|MethodCallTarget#HotSpotMethod<InlineCacheNode.evalPromise(Frame, Closure)>
properties:{invokeKind=Static, returnStamp=a Ljava/lang/Object;, targetMethod=HotSpotMethod<InlineCacheNode.evalPromise(Frame, Closure)>, profile=null, stamp=void}
arguments: [13766|NewFrame, 21786|LoadField#closure]
at com.oracle.truffle.r.nodes.InlineCacheNode.evalPromise(InlineCacheNode.java)
at com.oracle.truffle.r.nodes.InlineCacheNode.doGeneric(InlineCacheNode.java:77)
at com.oracle.truffle.r.nodes.InlineCacheNodeGen.execute(InlineCacheNodeGen.java:43)
at com.oracle.truffle.r.nodes.function.PromiseHelperNode.generateValueFromFullPromise(PromiseHelperNode.java:231)
at com.oracle.truffle.r.nodes.function.PromiseHelperNode.evaluateImpl(PromiseHelperNode.java:211)
at com.oracle.truffle.r.nodes.function.PromiseHelperNode.evaluate(PromiseHelperNode.java:173)
at com.oracle.truffle.r.nodes.function.PromiseHelperNode$PromiseCheckHelperNode.checkEvaluate(PromiseHelperNode.java:100)
at com.oracle.truffle.r.nodes.builtin.base.Switch.doSwitchString(Switch.java:128)
at com.oracle.truffle.r.nodes.builtin.base.Switch.doSwitch(Switch.java:109)
at com.oracle.truffle.r.nodes.builtin.base.SwitchNodeGen.execute(SwitchNodeGen.java:39)
at com.oracle.truffle.r.nodes.builtin.RBuiltinNode$Arg2.call(RBuiltinNode.java:187)
at com.oracle.truffle.r.nodes.function.RCallNode$BuiltinCallNode.execute(RCallNode.java:1149)
at com.oracle.truffle.r.nodes.function.RCallNode$FunctionDispatch.dispatch(RCallNode.java:911)
at com.oracle.truffle.r.nodes.function.RCallNodeGen$FunctionDispatchNodeGen.execute(RCallNodeGen.java:859)
at com.oracle.truffle.r.nodes.function.RCallNode.call(RCallNode.java:289)
at com.oracle.truffle.r.nodes.function.RCallNodeGen.execute(RCallNodeGen.java:178)
at com.oracle.truffle.r.runtime.nodes.RNode.visibleExecute(RNode.java:74)
at com.oracle.truffle.r.nodes.control.BlockNode.visibleExecute(BlockNode.java:119)
at com.oracle.truffle.r.nodes.function.FunctionBodyNode.visibleExecute(FunctionBodyNode.java:58)
at com.oracle.truffle.r.nodes.function.FunctionDefinitionNode.execute(FunctionDefinitionNode.java:282)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callProxy(OptimizedCallTarget.java:327)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callInlined(OptimizedCallTarget.java:267)
at org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:63)
at com.oracle.truffle.r.nodes.function.call.CallRFunctionNode.execute(CallRFunctionNode.java:63)
at com.oracle.truffle.r.nodes.function.RCallNode$DispatchedCallNode.execute(RCallNode.java:1213)
at com.oracle.truffle.r.nodes.function.RCallNode$FunctionDispatch.dispatch(RCallNode.java:911)
at com.oracle.truffle.r.nodes.function.RCallNodeGen$FunctionDispatchNodeGen.execute(RCallNodeGen.java:859)
at com.oracle.truffle.r.nodes.function.RCallNode.callInternalGeneric(RCallNode.java:371)
at com.oracle.truffle.r.nodes.function.RCallNodeGen.execute(RCallNodeGen.java:186)
at com.oracle.truffle.r.runtime.nodes.RNode.visibleExecute(RNode.java:74)
at com.oracle.truffle.r.nodes.control.BlockNode.visibleExecute(BlockNode.java:119)
at com.oracle.truffle.r.nodes.function.FunctionBodyNode.visibleExecute(FunctionBodyNode.java:58)
at com.oracle.truffle.r.nodes.function.FunctionDefinitionNode.execute(FunctionDefinitionNode.java:282)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callProxy(OptimizedCallTarget.java:327)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callRoot(OptimizedCallTarget.java:317)
Caused by: org.graalvm.compiler.graph.VerificationError: Frame escapes at: 21788|MethodCallTarget#HotSpotMethod<InlineCacheNode.evalPromise(Frame, Closure)>
properties:{invokeKind=Static, returnStamp=a Ljava/lang/Object;, targetMethod=HotSpotMethod<InlineCacheNode.evalPromise(Frame, Closure)>, profile=null, stamp=void}
arguments: [13766|NewFrame, 21786|LoadField#closure]
at org.graalvm.compiler.truffle.compiler.phases.VerifyFrameDoesNotEscapePhase.run(VerifyFrameDoesNotEscapePhase.java:47)
at org.graalvm.compiler.phases.Phase.run(Phase.java:49)
at org.graalvm.compiler.phases.BasePhase.apply(BasePhase.java:197)
at org.graalvm.compiler.phases.Phase.apply(Phase.java:42)
at org.graalvm.compiler.truffle.compiler.PartialEvaluator.createGraph(PartialEvaluator.java:253)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:441)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:637)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:579)
at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:178)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:280)
at org.graalvm.compiler.truffle.compiler.hotspot.libgraal.HotSpotToSVMEntryPoints.doCompile(HotSpotToSVMEntryPoints.java:253)
[truffle] opt fail @369f0a63 |Reason jdk.vm.ci.code.BailoutException: Code installation failed: code is too large
jdk.vm.ci.code.BailoutException: Code installation failed: code is too large
at jdk.vm.ci.hotspot.HotSpotCodeCacheProvider.installCode(HotSpotCodeCacheProvider.java:150)
at org.graalvm.compiler.core.target.Backend.createInstalledCode(Backend.java:194)
at org.graalvm.compiler.core.target.Backend.createInstalledCode(Backend.java:147)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compilePEGraph(TruffleCompilerImpl.java:534)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.compileAST(TruffleCompilerImpl.java:455)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:637)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl$TruffleCompilationWrapper.performCompilation(TruffleCompilerImpl.java:579)
at org.graalvm.compiler.core.CompilationWrapper.run(CompilationWrapper.java:178)
at org.graalvm.compiler.truffle.compiler.TruffleCompilerImpl.doCompile(TruffleCompilerImpl.java:280)
at org.graalvm.compiler.truffle.compiler.hotspot.libgraal.HotSpotToSVMEntryPoints.doCompile(HotSpotToSVMEntryPoints.java:253)
Thanks Fabio for the input.
I think we'll wait for @chumer to come back from vacation to discuss the Code installation
issue further. It would be useful if you could share with us the code for reproduction.
The Frame escape
is a different and a new issue. I think that the attached patch, which we will integrate soon, is going to fix it.
diff --git a/com.oracle.truffle.r.nodes/src/com/oracle/truffle/r/nodes/InlineCacheNode.java b/com.oracle.truffle.r.nodes/src/com/oracle/truffle/r/nodes/InlineCacheNode.java
index 2c39abd44..7f0c65d28 100644
--- a/com.oracle.truffle.r.nodes/src/com/oracle/truffle/r/nodes/InlineCacheNode.java
+++ b/com.oracle.truffle.r.nodes/src/com/oracle/truffle/r/nodes/InlineCacheNode.java
@@ -29,6 +29,7 @@ import com.oracle.truffle.api.dsl.Cached;
import com.oracle.truffle.api.dsl.ReportPolymorphism;
import com.oracle.truffle.api.dsl.Specialization;
import com.oracle.truffle.api.frame.Frame;
+import com.oracle.truffle.api.frame.MaterializedFrame;
import com.oracle.truffle.api.frame.VirtualFrame;
import com.oracle.truffle.api.nodes.Node;
import com.oracle.truffle.api.profiles.ConditionProfile;
@@ -74,7 +75,7 @@ public abstract class InlineCacheNode extends RBaseNode {
@Specialization(replaces = "doCached")
protected Object doGeneric(Frame frame, Object value) {
- return evalPromise(frame, (Closure) value);
+ return evalPromise(frame.materialize(), (Closure) value);
}
protected RNode cache(Object value) {
@@ -83,7 +84,7 @@ public abstract class InlineCacheNode extends RBaseNode {
/**
* Creates an inline cache that will execute promises closures by using a PIC and falling back
- * to {@link InlineCacheNode#evalPromise(Frame, Closure)}.
+ * to {@link InlineCacheNode#evalPromise(MaterializedFrame, Closure)}.
*
* @param maxPicDepth maximum number of entries in the polymorphic inline cache
*/
@@ -92,8 +93,8 @@ public abstract class InlineCacheNode extends RBaseNode {
}
@TruffleBoundary
- protected static Object evalPromise(Frame frame, Closure closure) {
- return closure.eval(frame.materialize());
+ protected static Object evalPromise(MaterializedFrame frame, Closure closure) {
+ return closure.eval(frame);
}
protected class NodeInsertedClosure {
That makes sense. Anyway, so far I haven't seen any of these errors when running with your proposed flags. I'll keep you updated.
Does using the flag -Dgraal.TruffleFunctionInlining=false
can come with performance penalty?
Does using the flag -Dgraal.TruffleFunctionInlining=false can come with performance penalty?
Very likely...inlining enables many other JIT optimizations. But Truffle 19.3.0
comes with a new inlining heuristic (enable with -Dgraal.TruffleLanguageAgnosticInlining=true
), which might fix the issues above.
I've been running a NodeJS app that integrates R's ggplot on GraalVM CE 19.1.1 for a few hours and I get a lot of "Code installation failed: code is too large" opt failures (see below). Are these considered bugs or can they be ignored? If they are bugs, how can I help debugging them?
Fabio