search

JuliaTesting / ReTest.jl

Testing framework for Julia
Other
111 stars 6 forks source link

Clarify intention behind failing tests #48

Open danielsoutar opened 2 years ago

danielsoutar commented 2 years ago

If a developer were to run the tests for ReTest.jl, they would observe a number of tests failing, such as those found in Main.FailingLoops. While there is a good reason for having failing tests in ReTest.jl, namely checking the library behaves as expected when the tests fail, it is confusing/misleading for the developer, especially newcomers.

It would be good to indicate which tests are expected to fail. This would be most easily achieved by moving the failing tests into a dedicated file and includeing it into the main runtests.jl file. Alternatively, we could potentially mark these with test_broken, although the Julia docs state that this macro is for indicating "a test that should pass".

rfourquet commented 2 years ago

Yes this is a good point. Isn't it ironic that the test suite for ReTest looks so bad ;-) I've definitely focused on improving ReTest itself rather than having its own tests look nice, but it would definitely be great to improve on that front too. This isn't a priority though.