Set up continuous benchmarking

[jcrist]

msgspec is a performance oriented library - would be useful to set up continuous benchmarking as part of our CI to catch any performance regressions. These might make use of airspeed velocity, or something else.

[jcrist]

A few links that might be useful:

• https://labs.quansight.org/blog/2021/08/github-actions-benchmarks
• https://asv.readthedocs.io/en/stable/commands.html#asv-continuous
• https://github.com/conbench/conbench

[jcrist]

I've seen some libraries are using CodSpeed (https://codspeed.io/) for this now. I'm slightly hesitant to rely on a new closed-source service.

From reading their Python client code, it looks like they're using callgrind to handle instrumentation. Depending on metrics used this may be a nice way to eliminate variation between runs in a public CI environment. Worth considering at least.

[jcrist]

Another potentially useful reference: https://pythonspeed.com/articles/consistent-benchmarking-in-ci/