Shopify / yjit-bench

Set of benchmarks for the YJIT CRuby JIT compiler and other Ruby implementations.
MIT License
87 stars 22 forks source link

Get the max RSS from getrusage(2) #248

Closed eregon closed 1 year ago

eregon commented 1 year ago

This is using Fiddle as that works on CRuby, JRuby and TruffleRuby (FFI is not shipped with CRuby, JRuby does not support C extensions and loading a C extension for this would be heavy at least on TruffleRuby).

This max RSS stat is a lot more stable than RSS and tracks the maximum memory usage, for example here are some runs on TruffleRuby:

# 1
RSS: 1065.4MiB
MAXRSS: 1631.9MiB
# 2
RSS: 810.3MiB
MAXRSS: 1603.3MiB
# 3
RSS: 1210.5MiB
MAXRSS: 1588.4MiB

On CRuby they are closer in these measurements, but RSS might still be brittle as e.g. a GC might happen "by luck" during the last iteration:

# 1
RSS: 558.5MiB
MAXRSS: 565.2MiB
# 2
RSS: 559.6MiB
MAXRSS: 566.3MiB

I also chatted about this with @noahgibbs a while ago, but finally got some clear numbers to show the value of it.

eregon commented 1 year ago

I am unsure if this maxrss stat should replace the existing rss one, so in doubt I kept both. I think it might make sense to replace the RSS one since it's so unstable on non-CRuby, and potentially brittle on CRuby (+ might not account for the JIT overhead correctly).

noahgibbs commented 1 year ago

This looks very reasonable to me, especially because by tracking both we can make sure we're not seeing any distressing results or unexpected large disparities. @maximecb, does this look reasonable to you?