psf / pyperf

Toolkit to run Python benchmarks
http://pyperf.readthedocs.io/
MIT License
771 stars 74 forks source link

Add hooks for calling arbitrary code around each benchmark #193

Closed mdboom closed 1 day ago

mdboom commented 1 month ago

This change adds a concept of "hooks" to pyperf for running arbitrary code before and after the actual internal benchmarking code. These are implemented with entry points, so it should be possible to write an external plugin to add new integrations and functionality.

There are a few real use cases that the Faster CPython team already needs this for:

The pystats integration that already exists in pyperf also should have, in hindsight, been implemented this way, so this PR also refactors things to do that, and it serves as a proof-of-concept of the new hook functionality.

~This PR needs tests but I thought I would submit a draft first to ensure this seems like the right direction first.~

mdboom commented 1 month ago

Would it be possible to use importlib.metadata instead of setuptools pkg_resources? See: https://docs.python.org/dev/library/importlib.metadata.html#entry-points

Sure -- I've never used entry points before -- I hadn't realized that's the new canonical way.