exercism / go-test-runner

GNU Affero General Public License v3.0
15 stars 17 forks source link

Test order should not be randomized #46

Closed angelikatyborska closed 2 years ago

angelikatyborska commented 2 years ago

I noticed that on every test run I am getting a slightly different test order. This might be fine for practice exercise (or not, I think that depends on the track), but it's not fine for concept exercises IMO. Those are meant to be solved task by task, so I always expect the test failures to be ordered by task. The tests for this exercise are ordered by task. It's the test runner that randomizes their order.

Run 1

Screenshot 2021-10-02 at 19-18-47 Exercism

Run 2

Screenshot 2021-10-02 at 19-19-13 Exercism

PS: There is a different issue visible on those screenshot (no tests for CalculateProductionRatePerMinute). I will debug/report that separately.

junedev commented 2 years ago

I am working on the fix.