Closed ntkme closed 2 years ago
Thank you for the report.
So that branch does rb_call_super() and that then likely calls BasicObject#method_missing (if it didn't find a super method, it would also be a NoMethodError FWIW).
I guess the main question is, as you say, why extract_method_call
sometimes returns METHOD_UNKNOWN
on TruffleRuby.
I guess the main question is, as you say, why
extract_method_call
sometimes returnsMETHOD_UNKNOWN
on TruffleRuby.
Exactly. I tried again with an iteration counter for the loop. This time, the counter shows truffleruby raised at 54th attempt (about 2% chance of running this rspec). For the roughly same amount of time, MRI ran for 851 iterations and never raised.
I managed to get a much faster reproduction of the issue:
gem install sass-embedded
ruby -r sass-embedded -e 'i = 0; loop do; Sass.compile_string(""); i += 1; puts i; end'
This usually reproduces NoMethodError
within a minute or two. However, at the same time it revealed two different bugs that happened randomly on truffleruby but not MRI:
Error while formatting Ruby exception:
org.truffleruby.core.rope.CannotConvertBinaryRubyStringToJavaString: Cannot convert a Ruby String with BINARY encoding containing non-US-ASCII character 208 to a Java String
at org.truffleruby.core.rope.RopeOperations.decodeNonAscii(RopeOperations.java:184)
at org.truffleruby.core.rope.RopeOperations.decodeRopeSegment(RopeOperations.java:241)
at org.truffleruby.core.rope.RopeOperations.decodeRopeSegment(RopeOperations.java:234)
at org.truffleruby.core.rope.RopeOperations.decodeRope(RopeOperations.java:229)
at org.truffleruby.core.string.RubyString.getJavaString(RubyString.java:87)
at org.truffleruby.core.string.RubyStringGen$RubyStringLibraryExports$Uncached.getJavaString(RubyStringGen.java:346)
at org.truffleruby.language.library.RubyStringLibraryGen$UncachedDispatch.getJavaString(RubyStringLibraryGen.java:459)
at org.truffleruby.language.backtrace.BacktraceFormatter.printRubyExceptionOnEnvStderr(BacktraceFormatter.java:137)
at org.truffleruby.language.backtrace.BacktraceFormatter.printTopLevelRubyExceptionOnEnvStderr(BacktraceFormatter.java:109)
at org.truffleruby.language.exceptions.TopLevelRaiseHandler.execute(TopLevelRaiseHandler.java:59)
at org.truffleruby.language.TruffleBootNodes$MainNode.main(TruffleBootNodes.java:123)
at org.truffleruby.language.TruffleBootNodesFactory$MainNodeFactory$MainNodeGen.executeAndSpecialize(TruffleBootNodesFactory.java:364)
at org.truffleruby.language.TruffleBootNodesFactory$MainNodeFactory$MainNodeGen.execute(TruffleBootNodesFactory.java:345)
at org.truffleruby.language.RubyCoreMethodRootNode.execute(RubyCoreMethodRootNode.java:48)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:655)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:627)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:560)
at com.oracle.svm.truffle.api.SubstrateOptimizedCallTarget.invokeCallBoundary(SubstrateOptimizedCallTarget.java:121)
at com.oracle.svm.truffle.api.SubstrateOptimizedCallTargetInstalledCode.doInvoke(SubstrateOptimizedCallTargetInstalledCode.java:176)
at com.oracle.svm.truffle.api.SubstrateOptimizedCallTarget.doInvoke(SubstrateOptimizedCallTarget.java:104)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callDirect(OptimizedCallTarget.java:486)
at org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:71)
at org.truffleruby.language.methods.CallInternalMethodNode.callCached(CallInternalMethodNode.java:61)
at org.truffleruby.language.methods.CallInternalMethodNodeGen.executeAndSpecialize(CallInternalMethodNodeGen.java:140)
at org.truffleruby.language.methods.CallInternalMethodNodeGen.execute(CallInternalMethodNodeGen.java:85)
at org.truffleruby.language.dispatch.DispatchNode.dispatch(DispatchNode.java:138)
at org.truffleruby.language.dispatch.RubyCallNode.executeWithArgumentsEvaluated(RubyCallNode.java:140)
at org.truffleruby.language.dispatch.RubyCallNode.execute(RubyCallNode.java:101)
at org.truffleruby.language.control.SequenceNode.execute(SequenceNode.java:36)
at org.truffleruby.language.RubyLambdaRootNode.execute(RubyLambdaRootNode.java:89)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:655)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:627)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:560)
at com.oracle.svm.truffle.api.SubstrateOptimizedCallTarget.invokeCallBoundary(SubstrateOptimizedCallTarget.java:121)
at com.oracle.svm.truffle.api.SubstrateOptimizedCallTargetInstalledCode.doInvoke(SubstrateOptimizedCallTargetInstalledCode.java:176)
at com.oracle.svm.truffle.api.SubstrateOptimizedCallTarget.doInvoke(SubstrateOptimizedCallTarget.java:104)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callDirect(OptimizedCallTarget.java:486)
at org.graalvm.compiler.truffle.runtime.OptimizedDirectCallNode.call(OptimizedDirectCallNode.java:71)
at org.truffleruby.language.yield.CallBlockNode.callBlockCached(CallBlockNode.java:57)
at org.truffleruby.language.yield.CallBlockNodeGen.executeAndSpecialize(CallBlockNodeGen.java:91)
at org.truffleruby.language.yield.CallBlockNodeGen.executeCallBlock(CallBlockNodeGen.java:51)
at org.truffleruby.language.yield.CallBlockNode.yield(CallBlockNode.java:41)
at org.truffleruby.core.proc.RubyProc.execute(RubyProc.java:137)
at org.truffleruby.core.proc.RubyProcGen$InteropLibraryExports$Cached.executeAndSpecialize(RubyProcGen.java:118)
at org.truffleruby.core.proc.RubyProcGen$InteropLibraryExports$Cached.execute(RubyProcGen.java:104)
at com.oracle.truffle.api.interop.InteropLibraryGen$CachedDispatch.execute(InteropLibraryGen.java:7588)
at com.oracle.truffle.polyglot.PolyglotValueDispatch$InteropValue$AbstractExecuteNode.executeShared(PolyglotValueDispatch.java:4234)
at com.oracle.truffle.polyglot.PolyglotValueDispatch$InteropValue$ExecuteNode.executeImpl(PolyglotValueDispatch.java:4315)
at com.oracle.truffle.polyglot.HostToGuestRootNode.execute(HostToGuestRootNode.java:125)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.executeRootNode(OptimizedCallTarget.java:655)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.profiledPERoot(OptimizedCallTarget.java:627)
at org.graalvm.compiler.truffle.runtime.OptimizedCallTarget.callBoundary(OptimizedCallTarget.java:560)
at com.oracle.svm.truffle.api.SubstrateOptimizedCallTarget.invokeCallBoundary(SubstrateOptimizedCallTarget.java:121)
at com.oracle.svm.truffle.api.SubstrateOptimizedCallTargetInstalledCode.doInvoke(SubstrateOptimizedCallTargetInstalledCode.java:176)
at com.oracle.svm.truffle.api.SubstrateOptimizedCallTarget.doInvoke(SubstrateOptimizedCallTarget.java:104)
at org.graalvm.compiler.truffle.runtime.GraalRuntimeSupport.callProfiled(GraalRuntimeSupport.java:235)
at com.oracle.truffle.polyglot.PolyglotValueDispatch$InteropValue.execute(PolyglotValueDispatch.java:2372)
at org.graalvm.polyglot.Value.execute(Value.java:841)
at org.truffleruby.launcher.RubyLauncher.runContext(RubyLauncher.java:266)
at org.truffleruby.launcher.RubyLauncher.runRubyMain(RubyLauncher.java:214)
at org.truffleruby.launcher.RubyLauncher.launch(RubyLauncher.java:126)
at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:295)
at org.graalvm.launcher.AbstractLanguageLauncher.launch(AbstractLanguageLauncher.java:121)
at org.truffleruby.launcher.RubyLauncher.main(RubyLauncher.java:38)
Original Ruby exception:
(exception org.truffleruby.core.rope.CannotConvertBinaryRubyStringToJavaString Cannot convert a Ruby String with BINARY encoding containing non-US-ASCII character 208 to a Java String org.truffleruby.core.rope.RopeOperations.decodeNonAscii(RopeOperations.java:184)
from exception.c:29:in `rb_exc_raise'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/message.c:598:in `Message_initialize_kwarg'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:949:in `execute_without_conversion'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:949:in `block in rb_hash_foreach'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:948:in `each'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:948:in `rb_hash_foreach'
from hash.c:100:in `rb_hash_foreach'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/message.c:610:in `Message_InitFromValue'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/message.c:643:in `Message_initialize'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext_ruby.rb:41:in `initialize'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:46:in `block in compile_request'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:129:in `block in await'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:124:in `synchronize'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:124:in `await'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:38:in `compile_request'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded.rb:166:in `compile_string'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass.rb:27:in `compile_string'
from -e:1:in `block in <main>'
from <internal:core> core/kernel.rb:407:in `loop'
from -e:1:in `<main>'
/opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:1086:in `rb_exc_raise': Unknown field name '' in initialization map entry. (ArgumentError)
from exception.c:29:in `rb_exc_raise'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/message.c:598:in `Message_initialize_kwarg'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:949:in `execute_without_conversion'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:949:in `block in rb_hash_foreach'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:948:in `each'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:948:in `rb_hash_foreach'
from hash.c:100:in `rb_hash_foreach'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/message.c:610:in `Message_InitFromValue'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/message.c:643:in `Message_initialize'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext_ruby.rb:41:in `initialize'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:46:in `block in compile_request'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:129:in `block in await'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:124:in `synchronize'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:124:in `await'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded/host.rb:38:in `compile_request'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded.rb:166:in `compile_string'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass.rb:27:in `compile_string'
from -e:1:in `block in <main>'
from <internal:core> core/kernel.rb:407:in `loop'
from -e:1:in `<main>'
It turns out that truffleruby even randomly crashes when loading protobuf library and definitions:
/opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:1086:in `rb_exc_raise': Unknown symbol value for enum field 'label'. (RangeError)
from exception.c:29:in `rb_exc_raise'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/convert.c:124:in `Convert_RubyToUpb'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/message.c:571:in `Message_initialize_kwarg'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:949:in `execute_without_conversion'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:949:in `block in rb_hash_foreach'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:948:in `each'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:948:in `rb_hash_foreach'
from hash.c:100:in `rb_hash_foreach'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/message.c:610:in `Message_InitFromValue'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/ext/google/protobuf_c/message.c:643:in `Message_initialize'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext_ruby.rb:41:in `initialize'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/lib/google/protobuf/descriptor_dsl.rb:369:in `internal_add_field'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/lib/google/protobuf/descriptor_dsl.rb:305:in `repeated'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/ext/sass/embedded_sass_pb.rb:224:in `block (3 levels) in <top (required)>'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/lib/google/protobuf/descriptor_dsl.rb:79:in `instance_eval'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/lib/google/protobuf/descriptor_dsl.rb:79:in `add_message'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/ext/sass/embedded_sass_pb.rb:223:in `block (2 levels) in <top (required)>'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/lib/google/protobuf/descriptor_dsl.rb:35:in `instance_eval'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/lib/google/protobuf/descriptor_dsl.rb:35:in `add_file'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/ext/sass/embedded_sass_pb.rb:7:in `block in <top (required)>'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/lib/google/protobuf/descriptor_dsl.rb:453:in `instance_eval'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/google-protobuf-3.19.4/lib/google/protobuf/descriptor_dsl.rb:453:in `build'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/ext/sass/embedded_sass_pb.rb:6:in `<top (required)>'
from <internal:core> core/kernel.rb:293:in `require_relative'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass/embedded.rb:4:in `<top (required)>'
from <internal:core> core/kernel.rb:293:in `require_relative'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass.rb:3:in `<top (required)>'
from <internal:core> core/kernel.rb:293:in `require_relative'
from /opt/truffleruby-22.0.0.2-linux-amd64/lib/gems/gems/sass-embedded-1.0.20/lib/sass-embedded.rb:4:in `<top (required)>'
from <internal:core> core/kernel.rb:234:in `gem_original_require'
from <internal:/opt/truffleruby-22.0.0.2-linux-amd64/lib/mri/rubygems/core_ext/kernel_require.rb>:160:in `require'
from <internal:core> core/unbound_method.rb:18:in `bind_call'
from <internal:core> core/kernel.rb:272:in `require'
Thanks, those smaller reproductions will be helpful.
Are you using native image @ntkme? I've tried this on both master, and the at 22.0.0.2 and have successfully done 100,000 iterations of the test spec without an error running in JVM mode. A dump of your environment variables might also be useful (especially LOCALE) just in case we've got something going wrong with a string overflowing a buffer, or something similar.
I'm using official truffleruby container image: ghcr.io/graalvm/truffleruby:22.0.0.2
. All environment variables are default. Specifically, LANG=en_US.UTF-8
.
$ uname -a
Linux penguin 5.10.92-14532-g179c52887ab5 #1 SMP PREEMPT Tue Mar 1 18:42:32 PST 2022 x86_64 GNU/Linux
In my environment the following usually crashes before hitting 10000 iterations, and the issues are reliably reproducible:
gem install sass-embedded
ruby -r sass-embedded -e 'i = 0; loop do; Sass.compile_string(""); i += 1; puts i; end'
Please let me know if there is any other information I can get to help debug this.
To make sure this is not just my local environment, I created a VM on GCP with Container Optimized OS and run the test mentioned in the previous comment. I was able to get it consistently reproduced on the GCP VM.
Ah, yes. I've managed to reproduce it with 22.0.0.2 in native image mode. I haven't been able to reproduce it on master in native image mode, so I think it's been fixed, but I haven't bisected it fully to work out which change fixed the problem.
This has now been fixed on master. The problem was that rb_id2name
generated a string which was not held onto by anything, so the RSTRING_PTR
from that (which was returned to the caller) could be freed while the caller was still using it.
Truffleruby raises several different kinds of exceptions from
google-protobuf
gem where MRI works without issue.NoMethodError
org.truffleruby.core.rope.CannotConvertBinaryRubyStringToJavaString: Cannot convert a Ruby String with BINARY encoding containing non-US-ASCII character 208 to a Java String
/opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:1086:in `rb_exc_raise': Unknown field name '' in initialization map entry. (ArgumentError)
/opt/truffleruby-22.0.0.2-linux-amd64/lib/truffle/truffle/cext.rb:1086:in `rb_exc_raise': Unknown symbol value for enum field 'label'. (RangeError)
See below for details of the first error. See comments below for other errors.
The error was from this piece of C code. Basically, protobuf overloads
Kernel#method_missing
in itsMessage
class in order to dynamically resolve and execute methods. When resolving the method in truffleruby there is a very low chance it incorrect went intoaccessor_type == METHOD_UNKNOWN
branch for unknown reason, and cause a unexpectedNoMethodError
.Reproduction: