Add a built-in benchmark tool

[dhh]

We should have a simple, built-in benchmark mechanism for people to evaluate different computing platforms. Whether they be cloud VMs, Raspberry Pis, or 96-core state-of-the-art owned server hardware. Think Geekbench or Speedometer. Run a big test suite, on a single machine, produce one comparable number. "Oh, that new box we got? It does 3,456 on the Rails Speedrun!!".

Easily comparable numbers make for better comparisons make for better buying decisions.

[nickjj]

I really like this, what do you envision the scope of this being?

For example:

• A clone'able demo app that Rails maintains which has all of these tests set up and ready to go where someone interested in comparing these numbers could clone the repo on a box and run a command to run the benchmarks?
• A new library within Rails that makes it easy to write custom performance tests and benchmarks for your custom app
• Something else?

The first option is interesting because it could be used to help create "real world'ish" performance regression tests for Rails itself.

[dhh]

I'm thinking this is a gem that includes a very basic test controller that talks to the database with both reading and writing. Then have something like an ab or whatever that hits that test controller. Then produces a combined score. It should not be very complicated. Please do investigate this!

[chadbrewbaker]

My favorite is https://github.com/sirupsen/napkin-math - can apply most of those benchmarks to any system.

Using Halvarflake's profiler is informative https://github.com/open-telemetry/community/issues/1918

Use the TPC-H schema/data/queries?

Front end a lot of the problem is zscaler and Microsoft AD auth hell for enterprises. Auth benchmark very useful.

With CPU cores now approaching 256 and non-uniform P/E cores, the topology of how you allocate them is non-trivial. This to me is the more interesting benchmark. When I ported Ruby to Blue Gene/L there were only two cores so this wasn't an issue and I was using IBM's tuned topology for collectives. https://github.com/open-mpi/ompi/issues/11345

On linux I would zoom out and think about what made the IBM Blue Gene/L so fast - it stripped everything out of the kernel you didn't need to reduce jitter. Rails can emit a linux kernel config optimized to the application for production use.

Benchmark OS page sizes and using zram. https://docs.kernel.org/admin-guide/blockdev/zram.html

COZ profiling - experimentally slowing code to see what is causing bottlenecks. https://github.com/plasma-umass/coz

• [ ] quick and dirty
• [ ] longer to get system jitter patterns
• [ ] cheat-mode where you break Ruby's third wall to build a custom kernel, optimize how processes are pinned to cores, use profile guided optimizations ...