search

hyperskill / hs-test

A framework that simplifies testing educational projects for Hyperskill. It is built on top of JUnit.
31 stars 10 forks source link

[Obsolete] Prohibit throwing Error in generateTests and check #35

Closed SerVB closed 5 years ago

SerVB commented 5 years ago

This resolves #32.

aaaaaa2493 commented 5 years ago

Why there are commits that were already merged? Update your fork and let me see actual changes

SerVB commented 5 years ago

Yeah, working on it. Marked as WIP, by the way ;)

SerVB commented 5 years ago

Please take a look.

aaaaaa2493 commented 5 years ago

Oh, I mistyped the issue description - I meant in check method, not solve. Because some external test writers use Assert instead of returning CheckResult and that is bad. So any AssertionError produced in check should be handled as Fatal error in test #N like any exception that is thrown during checking.

There can't be AssertionError in solve method simply because a method called using reflection and if there are any Throwable while running this method then InvocationTargetException would be raised (but I'm not 100% sure, it may be true only for Exception bit not for Error and you can try it).

Oh, now I noticed that I talked about run method above but not solve :) Still, I have plans to remove solve, checkSolved, generatePredefinedInputOutput for a better design.

Guess, the issue is transposed to be named Prohibit throwing Error in generateTests and check

aaaaaa2493 commented 5 years ago

And also, keep in mind that Fatal error should be generated here, in a single place (there is a uniform way of feedback with report and stacktrace, not just Fatal error like in your solution) https://github.com/hyperskill/hs-test/blob/dev/src/main/java/org/hyperskill/hstest/dev/exception/FailureHandler.java#L203

SerVB commented 5 years ago

Replaced by #39.