ColinEberhardt / assemblyscript-regex

A regex engine for AssemblyScript
MIT License
86 stars 12 forks source link

Run benchmark tests on node v15 #29

Closed ColinEberhardt closed 3 years ago

ColinEberhardt commented 3 years ago

Currently the CI tests (that run npm test) use node v15. However, the benchmarks use the default node version, which happens to be v10.

When running the benchmarks on node v15 they fail as follows:

$ npm run benchmark

> assemblyscript-regex@0.1.0 benchmark
> node benchmark/benchmark.js

baseline x 134,050 ops/sec ±0.79% (88 runs sampled)
character class x 43,542 ops/sec ±2.28% (87 runs sampled)
concatenation x 9,974 ops/sec ±1.11% (86 runs sampled)
/Users/colineberhardt/Projects/as-regex/node_modules/@assemblyscript/loader/umd/index.js:78
      throw Error(`abort: ${getString(memory, msg)} at ${getString(memory, file)}:${line}:${colm}`);
      ^

Error: abort:  at ~lib/rt/tlsf.ts:238:14
    at abort (/Users/colineberhardt/Projects/as-regex/node_modules/@assemblyscript/loader/umd/index.js:78:13)
    at ~lib/rt/tlsf/insertBlock (<anonymous>:wasm-function[2]:0x69f)
    at ~lib/rt/tlsf/addMemory (<anonymous>:wasm-function[3]:0x8e7)
    at ~lib/rt/tlsf/growMemory (<anonymous>:wasm-function[8]:0xc0c)
    at ~lib/rt/tlsf/allocateBlock (<anonymous>:wasm-function[10]:0xce1)
    at ~lib/rt/tlsf/__alloc (<anonymous>:wasm-function[11]:0xd46)
    at ~lib/rt/pure/__new (<anonymous>:wasm-function[12]:0xd72)
    at ~lib/rt/__newArray (<anonymous>:wasm-function[33]:0x1d7f)
    at assembly/regexp/recursiveBacktrackingSearch (<anonymous>:wasm-function[181]:0x5025)
    at assembly/regexp/recursiveBacktrackingSearch (<anonymous>:wasm-function[181]:0x507a)

Ideally everything should run on the same node version!

MaxGraey commented 3 years ago

I tried 0.18 version from this PR on node v15 and everything works fine:

untouched.wasm:

baseline x 103,084 ops/sec ±1.14% (88 runs sampled)
character class x 28,805 ops/sec ±0.52% (89 runs sampled)
concatenation x 8,267 ops/sec ±0.61% (91 runs sampled)
quantifiers x 15,565 ops/sec ±0.96% (86 runs sampled)
range quantifiers x 5,022 ops/sec ±0.56% (89 runs sampled)
alternation x 16,263 ops/sec ±0.63% (88 runs sampled)
multiple regex matches x 1,723 ops/sec ±0.44% (86 runs sampled)

optimized.wasm:

baseline x 205,100 ops/sec ±0.66% (89 runs sampled)
character class x 58,661 ops/sec ±0.76% (88 runs sampled)
concatenation x 15,636 ops/sec ±0.37% (89 runs sampled)
quantifiers x 30,233 ops/sec ±0.51% (88 runs sampled)
range quantifiers x 9,227 ops/sec ±0.39% (84 runs sampled)
alternation x 29,869 ops/sec ±0.70% (88 runs sampled)
multiple regex matches x 2,309 ops/sec ±1.03% (86 runs sampled)
torch2424 commented 3 years ago

Ah I was just about to try a benchmark, thanks for checking this @MaxGraey ! :smile: :tada:

ColinEberhardt commented 3 years ago

Fixed https://github.com/ColinEberhardt/assemblyscript-regex/commit/12c8e552264acea48cc5bf98ed4d2e606254e720