search

oracle / truffleruby

A high performance implementation of the Ruby programming language, built on GraalVM.
https://www.graalvm.org/ruby/
Other
2.98k stars 180 forks source link

`jt test fast` fails with JVM CE environment #3516

Closed nirvdrum closed 2 weeks ago

nirvdrum commented 3 months ago

In local development I run with JT_ENV=jvm-ce. I recently pulled from master and found that I had 12 test failures that I didn't expect. I thought maybe that I had some stale data, but running jt rebuild && jt test fast didn't fix the problem. Then I tried running again with JT_ENV=jvm and the test suite passed.

I only see the problem on macOS ARM64. I tested on Linux x86_64 and did not see the problem. I haven't bisected, but I think it's likely that #3508 is when the issue was introduced. 

$ /Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/mxbuild/truffleruby-jvm-ce/bin/ruby --vm.ea --vm.esa --experimental-options --core-load-path=/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/src/main/ruby/truffleruby --vm.XX:-UseJVMCICompiler --experimental-options --engine.Compilation=false --engine.Splitting=false --polyglot /Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/mspec/bin/mspec-run -B spec/truffleruby.mspec --excl-tag fails --excl-tag slow
truffleruby 24.1.0-dev-72f94f4c, like ruby 3.2.2, GraalVM CE JVM [aarch64-darwin]

1)
Inline caching for dynamically-created Regexp works for Regexp.new FAILED
Unexpected warning: 
"/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:29: warning: unbounded creation of regexps causes deoptimization loops which hurt performance significantly, avoid creating regexps dynamically where possible or cache them to fix this"
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:28:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

2)
Inline caching for dynamically-created Regexp works for Regexp.union with 1 argument FAILED
Unexpected warning: 
"/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:55: warning: unbounded creation of regexps causes deoptimization loops which hurt performance significantly, avoid creating regexps dynamically where possible or cache them to fix this"
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:51:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

3)
Inline caching for dynamically-created Regexp works for Regexp.union with multiple arguments FAILED
Unexpected warning: 
"/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:75: warning: unbounded creation of regexps causes deoptimization loops which hurt performance significantly, avoid creating regexps dynamically where possible or cache them to fix this"
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:74:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

4)
Inline caching for dynamically-created Regexp works for String#scan FAILED
Unexpected warning: 
"/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:107: warning: unbounded creation of regexps causes deoptimization loops which hurt performance significantly, avoid creating regexps dynamically where possible or cache them to fix this"
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:106:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

5)
Inline caching for dynamically-created Regexp works for String#sub FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:151:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

6)
Inline caching for dynamically-created Regexp works for String#sub! FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:181:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

7)
Inline caching for dynamically-created Regexp works for String#gsub FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:211:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

8)
Inline caching for dynamically-created Regexp works for String#gsub! FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:241:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

9)
Inline caching for dynamically-created Regexp works for String#match FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:264:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

10)
Inline caching for dynamically-created Regexp works for String#match? FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:287:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

11)
Inline caching for dynamically-created Regexp works for Symbol#match FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:310:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'

12)
Inline caching for dynamically-created Regexp works for Symbol#match? FAILED
Expected warning to match: /unbounded creation of regexps/ but got: ""
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:333:in `block (3 levels) in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:17:in `block in <top (required)>'
/Users/nirvdrum/dev/workspaces/truffleruby-ws/truffleruby/spec/truffle/regexp/inline_caching_spec.rb:16:in `<top (required)>'
[- | ==================100%================== | 00:00:00]     12F      0E
eregon commented 3 months ago

Do you have JT_SPECS_COMPILATION or JT_SPECS_SPLITTING set in the environment maybe? These specs do seem to pass fine in our darwin-aarch64 CI.

nirvdrum commented 3 months ago

I have JT_SPECS_COMPILATION enabled. When I remove that, the specs pass even on JVM CE.

eregon commented 3 months ago

OK, that's kind of expected then since JT_SPECS_COMPILATION=false does --engine.Splitting=false (unless JT_SPECS_SPLITTING=true). So we should check that condition in the guard then.

Ideally we would check specifically that splitting is enabled. I think that might be possible by calling DirectCallNode#isCallTargetCloned() on something we know we always-split like Array#[]. We could create a dummy DirectCallNode e.g. in CoreLibrary, call cloneCallTarget() and then if isCallTargetCloned() returns true it should mean splitting is enabled.

@andrykonchin Could you look into that?

andrykonchin commented 2 weeks ago

Should be fixed in 5603e8bcf5e35813e5cdff36f455079cf2aa1bc8.

cc @nirvdrum