search

byte-physics / igortest

Igor Pro Universal Testing Framework
https://docs.byte-physics.de/igor-unit-testing-framework/
BSD 3-Clause "New" or "Revised" License
7 stars 2 forks source link

Require UTF_EXPECTED_FAILURE to have errors #377

Closed Garados007 closed 1 year ago

Garados007 commented 1 year ago

Each test case that in now marked with UTF_EXPECTED_FAILURE has to create a failure or error at least once.

Close #289

Dependency

Garados007 commented 1 year ago

@t-b @MichaelHuth Ready for review.

t-b commented 1 year ago

Review:

Garados007 commented 1 year ago

Why do we need to set wvTestCase[%CURRENT][%NUM_ASSERT_ERROR] = "0" in AfterTestCase? This was not done before?

No. The updated implementation count errors as normal to allow the check 5 lines above this statement. After that we have to clear the error counter as these errors are considered as intended and an existing error counter is propagated upwards which let the whole test run fail.

Would it make sense to remove the logError parameter of TestCaseFail as you removed the last caller?

Thanks for the reminder. I have to update the documentation as well.

I will pause further reviews until PR #381 is merged. PR #381 has changed the location of some hooks which results in heavy rebases.

t-b commented 1 year ago

I will pause further reviews until PR https://github.com/byte-physics/igor-unit-testing-framework/pull/381 is merged. PR https://github.com/byte-physics/igor-unit-testing-framework/pull/381 has changed the location of some hooks which results in heavy rebases.

Okay.

t-b commented 1 year ago

When you rebase this PR please also have a look at the documentation at https://docs.byte-physics.de/igor-unit-testing-framework/basic.html?highlight=expected#test-case. That should mention the behaviour change that no failed assertions is a failure for expected failures.

Garados007 commented 1 year ago

@t-b @MichaelHuth Ready for next review round.