tc39 / eshost

A uniform wrapper around a multitude of ECMAScript hosts. CLI: https://github.com/bterlson/eshost-cli
Other
142 stars 37 forks source link

Can eshost provide cross-implementation benchmarking? #126

Open gibson042 opened 2 years ago

gibson042 commented 2 years ago

I sometimes find the need to compare code variations across implementations, for which eshost is actually a great supporting tool. I've got a script that hard-codes some loop controls and uses Date.now() for measurement, but it would be great to have (more sophisticated) functionality integrated here.

Interface sketch:

$ eshost -h V8,'*XS*' \
  --prelude '/* general option; runs before any expression or statement */
    "use strict";
    const { fromEntries } = Object;
    const { ownKeys } = Reflect;
    const reverseComparator = (left, right) => left < right ? 1 : left === right ? 0 : -1;
    const result;
  ' \
  --benchmark '/* mode switch and code to run before each timed section */
    const str = String.fromCodePoint(i).repeat(1000);
  ' \
  --benchmark--'str[0]' '
    /* named benchmark code */
    /* (`--benchmark--"$name" "$code"` structured option name form) */
    result = str[0];
  ' \
  --benchmark-named 'charAt(0)' '
    /* named benchmark code */
    /* (`--benchmark-named "$name" "$code"` two-argument option form) */
    result = str.charAt(0);
  ' \
  --benchmark-variant '"at(0)" '
    /* (optionally?)named benchmark code */
    /* (`--benchmark-variant "${<<<"$name" jq -R} $code"` space-separated structured option value form) */
    result = str.at(0);
  '
#### Moddable XS
str[0]: 1,445,376 ops/sec ±0.25% (91 runs sampled)
charAt(0): 2,992,670 ops/sec ±0.55% (85 runs sampled)
str.at(0): 1,417,580 ops/sec ±0.71% (90 runs sampled)

#### V8
str[0]: 225,206,729 ops/sec ±0.25% (91 runs sampled)
charAt(0): 138,581,640 ops/sec ±0.55% (85 runs sampled)
at(0): 62,485,375 ops/sec ±0.71% (90 runs sampled)