search

python / pyperformance

Python Performance Benchmark Suite
http://pyperformance.readthedocs.io/
MIT License
869 stars 175 forks source link

Impossible to add a benchmark that exercises newer Python features #281

Closed AlexWaygood closed 1 year ago

AlexWaygood commented 1 year ago

It appears to be currently impossible to add a benchmark that uses a feature added in Python >3.7. Even if you add the requires-python field in the pyproject.toml file for a specific benchmark, the benchmark is run on all Python versions in CI. This means that the CI will fail if the benchmark includes a feature that is new in 3.8, for example.

This issue currently blocks both of these open PRs:

brandtbucher commented 1 year ago

Yeah, pyperformance should probably check this somehow and refuse to run certain benchmarks based on the advertised Python version support.

Another option could be to disable them by default, and enable them when we drop older version support. But that seems less ideal.