psyinfra / onyo

text-based inventory system on top of git
ISC License
3 stars 5 forks source link

TST: add benchmarking and Faker support #733

Closed aqw closed 1 day ago

aqw commented 4 days ago

The core motivation for this PR is to add some benchmarks and test out CodSpeed to see if it's a reasonable fit for us.

If not, the benchmarks are still useful, but we'll go back to vanilla pytest-benchmark. Codspeed is mostly compatible, until it isn't. For example, it does not support pendantic, which is a loss.

Generating large, representative repos led me to finally add Faker support. I hope/plan to use this for more tests that require populated repos, rather than our current verbose, hard-coded mess.

This PR also:

Fixes:

I encountered numerous bugs around pytest fixture scoping, parametrization, indirect parametrization, and referring to the parent class.

As a result, I hack a class-level cache through a function-scoped fixture. I would like to to this differently, with a class-level fixture populating and function-level fixture cloning, but this was not to be.

I also setup Faker with fixtures for each scope. We may not use this, in which case we can remove it. But I wanted to include it for now so we have a history of it in the repo, and it may in fact turn out to be of use.

aqw commented 2 days ago

I have pushed a new set of commits to this PR.

Primary changes are:

If there are any questions on my approach, I am happy to explain. I'm not thrilled with the solution, but it was the best I could manage given the numerous pytest bugs I encountered.

aqw commented 1 day ago

Thanks for the review!