search

hauntsaninja / mypy_primer

Run mypy and pyright over millions of lines of code
MIT License
55 stars 29 forks source link

Compare run time to flag performance regressions #30

Closed huguesb closed 1 year ago

huguesb commented 2 years ago

It would be useful for mypy_primer to keep track of run times, compute a percent change, and report it when above a configurable threshold.

This would allow flagging performance regressions caused by mypy PRs, and could also validate the real-world impact of any optimizations.

Caveat: it might require more than 1 run of each branch to get reliable timings on CI

hauntsaninja commented 2 years ago

That's a great idea. Like you say, will depend on how stable timings are in CI.

hauntsaninja commented 1 year ago

I've added this to the mypy_primer output, we'll see how reliable the timings are