Shopify / yjit-bench

Set of benchmarks for the YJIT CRuby JIT compiler and other Ruby implementations.
MIT License
87 stars 22 forks source link

Print YJIT code size besides RSS #319

Closed k0kubun closed 3 months ago

k0kubun commented 3 months ago

This PR prints YJIT's memory-related stats besides RSS after each run of a benchmark.

When we check the YJIT code size on yjit-bench, we often use --yjit-stats and look at what it prints. However, because --yjit-stats generates more code than --yjit, it has some unwanted noise. To check the actual code size, we need to look at RubyVM::YJIT.runtime_stats without using --yjit, and this PR lets the harness do that.

Example:

$ ./run_benchmarks.rb lobsters --chruby 'ruby --yjit' --once
Running benchmark "lobsters" (1/1)
setarch x86_64 -R taskset -c 19 /opt/rubies/ruby/bin/ruby --yjit -I harness /home/k0kubun/src/github.com/Shopify/yjit-bench/benchmarks/lobsters/benchmark.rb
ruby 3.4.0dev (2024-08-15T17:34:25Z master de28ef7db4) +YJIT [x86_64-linux]
Command: bundle check 2> /dev/null || bundle install
The Gemfile's dependencies are satisfied
itr #1: 2779ms
RSS: 291.4MiB
MAXRSS: 291.4MiB
inline_code_size:    3,308,526
outlined_code_size:  2,873,627
code_region_size:    7,946,240
yjit_alloc_size:    18,478,455
Total time spent benchmarking: 9s

ruby: ruby 3.4.0dev (2024-08-15T17:34:25Z master de28ef7db4) +YJIT [x86_64-linux]

--------  ---------  ----------
bench     ruby (ms)  stddev (%)
lobsters  2779.7     0.0
--------  ---------  ----------

Output:
./data/output_1567.csv