donmccurdy / glTF-Transform

glTF 2.0 SDK for JavaScript and TypeScript, on Web and Node.js.
https://gltf-transform.dev
MIT License
1.42k stars 150 forks source link

Add benchmarks #1202

Closed donmccurdy closed 11 months ago

donmccurdy commented 11 months ago

Adds a small benchmark suite using tinybench, and sets up CI to monitor those benchmarks over time.

Related:

donmccurdy commented 11 months ago

Removing the CI (below) from the PR for now. Not fully happy with how the GitHub Action is managing that.

  #############################################################################
  # BENCHMARK
  benchmark:
    runs-on: ubuntu-latest
    env:
      NODE_VERSION: v18.18.x
    steps:
    # Setup.
    - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
    - uses: actions/checkout@b4ffde65f46336ab88eb53be808477a3936bae11 # v4
    - name: Use Node.js ${{ env.NODE_VERSION }}
      uses: actions/setup-node@v4
      with:
        node-version: ${{ env.NODE_VERSION }}
        cache: yarn

    # Build.
    - run: yarn install --frozen-lockfile
    - run: yarn dist

    # Benchmark.
    - run: yarn --silent benchmark | tee bench-results.json
    - name: Download benchmark archive
      uses: actions/cache@v3
      with:
        path: ./cache
        key: ${{ runner.os }}-benchmark
    - name: Upload benchmark results
      uses: benchmark-action/github-action-benchmark@v1
      with:
        tool: 'customSmallerIsBetter'
        output-file-path: bench-results.json
        external-data-json-path: ./cache/benchmark-data.json
        github-token: ${{ secrets.GITHUB_TOKEN }}
        summary-always: true
        fail-on-alert: true

I'll (later) look into having a benchmark run happen automatically on each release. Then each PR can compare against the last commit, and the last release, and we'll store that as a JSON file in the repository perhaps.