search

codewars / runner

Issue tracker for Code Runner
34 stars 8 forks source link

Test assertion for *passing* a test if it reaches a certain time limit #78

Open Voileexperiments opened 6 years ago

Voileexperiments commented 6 years ago

There are currently Test.assertNotEquals, Test.assertNotContains, Test.expectNoError... but there are no controls for the timeout. We can only make tests that fail on timeout, not succeed on timeout.

Use case: When there are 2 versions of a kata and one is the performance version, we'd like to block solutions that can pass the performance version from the easy version, so others can't just look for these solutions in the solution page and solve the performance version for free. @myjinxin2015's Coding 5min/Shortest Code series used to have this feature: if your code is short enough for the challenge version, submitting this code on the non-challenge version will cause tests to fail with the error message telling you to go submit your code in the challenge version instead.

Speaking of which, this also needs codewars/python-test-framework#4.

kazk commented 4 years ago

We can create a community maintained package for something like this. Let me know if anyone is interested to work on it.