search

donmccurdy / glTF-Transform

glTF 2.0 SDK for JavaScript and TypeScript, on Web and Node.js.
https://gltf-transform.dev
MIT License
1.43k stars 150 forks source link

Continuous benchmarking #1214

Closed donmccurdy closed 8 months ago

donmccurdy commented 11 months ago

Basic benchmarks were added to the project in https://github.com/donmccurdy/glTF-Transform/pull/1202. I'd like to set up continuous benchmarking, such that:

  1. [x] benchmark results are written to a CSV or JSON file
  2. [x] each release appends to archived results
  3. [x] branches and PRs can be compared to previous releases
  4. [ ] non-trivial regressions are flagged in PR review

I've reviewed existing GitHub actions and they don't fit my goals well, so this might be custom.

donmccurdy commented 8 months ago

Results from #1336, lower is better —

clone
v3.10.1             ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀     12.6604ms
v4.0.0-alpha.12     ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀     12.5776ms

create
v3.10.1             ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀     29.8105ms
v4.0.0-alpha.12     ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀      29.095ms

dequantize
v3.10.1             ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀     663.0726ms
v4.0.0-alpha.12     ▀                                            9.2248ms

dispose
v3.10.1             ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀     0.5123ms
v4.0.0-alpha.12     ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀      0.4989ms

join
v3.10.1             ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀     127.446ms
v4.0.0-alpha.12     ▀▀▀▀▀▀▀▀▀▀▀                                  33.6132ms

quantize
v3.10.1             ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀       95.8734ms
v4.0.0-alpha.12     ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀     100.5125ms

weld
v3.10.1             ▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀▀     575.7039ms
v4.0.0-alpha.12     ▀▀▀▀                                         64.7211ms