search

codewars / codewars-runner-cli

Old CodeRunner project. See https://github.com/codewars/runner instead.
GNU Affero General Public License v3.0
402 stars 141 forks source link

Compilation time is counted as running time, resulting in timeouts #788

Closed nomennescio closed 4 years ago

nomennescio commented 4 years ago

It seems to me that the running time of a Kata includes the times it takes to compile the code. I've noticed for Factor, that compilation for sample code can take 1.5-2 seconds, while actual execution takes less than a millisecond.

I've run into a Kata that will time-out for no obvious reason, but I suspect it has to do with the way the test code is structured, and its compilation time is counting towards timeouts. As it is a timeout error, I also am unable to see partial results, including time it takes to run particular tests. I can therefore not determine where the exact error is.

Either do not count compilation time as part of the user execution time for timeouts, or in case of a timeout, show more detailed information on the timing of the testcases already executed before the timeout.

kazk commented 4 years ago

We're not changing how maximum allowed time is measured.

To show more detail on the timeout, please open separate issues for the languages you had issues with because it all depends on the language.