search

sbi-benchmark / sbibm

Simulation-based inference benchmark
https://sbi-benchmark.github.io
MIT License
88 stars 34 forks source link

Improvements to unit tests #23

Closed psteinb closed 2 years ago

psteinb commented 2 years ago

While running into sbi API problems (see #21) I ran some unit tests locally on my laptop (mostly as a sanity check). This revealed very quickly that the unit tests, e.g. under algorithms, are geared towards correctness of results rather than correctness of API usage. I suspect that this is also the reason why this repo doesn't use CI facilities like github-actions.

While I value correctness of results very highly, the correctness of API is a precondition. If the code breaks before a result can be obtained, there will be no result to test for correctness. ;-) It might be worthwhile to introduce faster and slower tests (similar to what sbi does).

jan-matthis commented 2 years ago

Speed of the tests was one reason why I did not set them up on CI, but mostly I just post-poned it to the future, when we have more collaborators/the codebase is growing :)

Since this is the case now, I'll go through the tests, mark some as slow, and set things up on GH actions.