vipm-io / caraya

Assertion and unit test framework for LabVIEW
Other
59 stars 33 forks source link

TestRunner Linker Info error should preserve explanation #166

Closed francois-normandin closed 1 year ago

francois-normandin commented 1 year ago

Test Runner Linker Info error gets translated into but does not preserve the explanation in all cases. As an example, if a Test VI is saved in a more recent version, it would be difficult to find out why for VIs that are on disk and not in a project, class or being called at edit time.

image

Error 7101 occurred at At least one test is not in a executable state. Caraya.lvlib:Assert.lvclass:Assert_Core.vi:390020

Possible reason(s):

ASSERTATION FAILED: VI not in an executable state (C:\Repositories\JKI\Caraya\src\tests\test-runner\test-brokenTests\Broken Tests\Broken Test 0001.vi)

Complete call chain: Caraya.lvlib:Assert.lvclass:Assert_Core.vi:390020 Caraya.lvlib:Assert.lvclass:Assert Wrapper.vi:5770306 Caraya.lvlib:Assert.lvclass:Assert Not Error.vi:5190083 Test Runner.BrokenTest.lvclass:onTestExecution.vi:70001 Caraya.lvlib:Test Runner.lvclass:Run.vi:1530011 Test Batch Run with Broken Tests.vi

francois-normandin commented 1 year ago

Proposal is to extract errors from Read Link Info method (onTestDiscovery) and merge in the Error Out terminal. image This method is called by the static "Discover Tests" method, which then translated the errors into Caraya Error Codes (7xxx range). image

The immediate consequence of this change is that any Test Runner classes that were previously not expecting an error on the output terminal if one of the VI has a bad linkage error, would now error out before it is Run (at Discover Test). Until now, the error would be discovered when performing the test, not in the discovery.

I believe this is a bug, so correction is expected to change the behavior, but in a way that will catch previously uncaught issues during discovery.

image

francois-normandin commented 1 year ago

ASKING FOR COMMENTS.

francois-normandin commented 1 year ago

Please use discussion thread to discuss. Try it with 1.4.0 RC1 version https://github.com/JKISoftware/Caraya/discussions/172