devenjarvis / sundew

Making Python testing easy, enjoyable, and effective ☀️
https://devenjarvis.github.io/sundew/
MIT License
10 stars 1 forks source link

Introduce smart test sorting #11

Closed devenjarvis closed 1 year ago

devenjarvis commented 1 year ago

With these changes, we now run tests in order by least dependencies to most. This insures that all dependent functions a FUT (function under test) has, have already been tested (if that have tests) before we run tests for that function. This gives us a couple significant benefits:

  1. If a test fails because of a change to a dependent function, a well covered test suite will catch the error in that function before trying to run tests that depend on it. This gives us some implicit narrowing of where the bug is
  2. Due to the assertion that a function that calls another function can be no faster than the function it is calling, we should always be running the fastest tests first. This gives us a nice "fail fast" feedback loop, reducing the likelihood with large test suites that we have to wait a long time before we realize something is broken

I believe this is the first real step in unlocking unique benefits of function-based testing in Sundew 🎉