Unstable test runtime - same code behaves unpredictably

[wsgac]

I've recently noticed (based on two of my solutions) that code passing local tests instantly times out on your servers. Upon resubmitting the very same code passes. Below are links to said exercises:

• https://exercism.org/tracks/haskell/exercises/binary-search-tree/solutions/wsgac (successful on 2nd submission)
• https://exercism.org/tracks/haskell/exercises/allergies/solutions/wsgac (successful on 3rd submission) Also, there's no indication at what stage the test runner times out. It would certainly be more helpful if such information were available.

[cafce25]

Experiencing this as well, if you're sure your code should work you can rerun the tests by going into the online editor and clicking run tests there. This seems to be especially severe when the solution in question does have a slightly longer build time (additional dependencies) A solution to this could be building the tests with less strict timing requirements beforehand. (stack test --no-run-tests) I'd try to create a PR but can't seem to find the server side test running code anywhere.

[petertseng]

I'd try to create a PR but can't seem to find the server side test running code anywhere.

That sounds like a deficiency in the track's documentation. I have created https://github.com/exercism/haskell/issues/1056 to ask for this deficiency to be corrected.

[cafce25]

Yea adding it to the readme should fix that part. I've dug around a little more since I last wrote and it seems to me this cannot currently be fixed since the timeout is enforced on a higher layer (the respective docker container is killed) See https://github.com/exercism/exercism/issues/6238 and the documentation for tooling