search

smarr / are-we-fast-yet

Are We Fast Yet? Comparing Language Implementations with Objects, Closures, and Arrays
Other
318 stars 36 forks source link

Should frozen_string_literal be used for Ruby benchmarks? #72

Closed eregon closed 1 year ago

eregon commented 1 year ago

For instance on Json it seems like it would make sense given there are lots of String literals, which are actually allocations without # frozen_string_literal: true as first line. From a quick local run, it's faster on CRuby with frozen_string_literal and it is about the same for TruffleRuby:

CRuby: ~428091 -> ~290305
TruffleRuby: 66057us -> 65282us

OTOH, maybe we want to measure how well the VM optimizes?

Ruby code in the wild is a mix of # frozen_string_literal: true and not using it. Ruby 3 still isn't using that by default over compatibility concerns. I'd think nowadays for performance-sensitive files people typically use # frozen_string_literal: true.

smarr commented 1 year ago

Hmmm. So, I did actually add those to the copy I am using for tracking TruffleRuby performance.

https://github.com/smarr/truffleruby-faststart/commit/f2a232426ca6d399a4f7d61799fb893b5f89ed1b

Not sure why I didn't propagate it back here.

eregon commented 1 year ago

Right, at least for interpreter performance I think adding it makes a lot of sense. Could you apply it to this repo too?