byte-physics / igortest

Igor Pro Universal Testing Framework
https://docs.byte-physics.de/igor-unit-testing-framework/
BSD 3-Clause "New" or "Revised" License
7 stars 2 forks source link

Tracing.pxp fails locally #432

Closed t-b closed 1 year ago

t-b commented 1 year ago

Version: 977046d4 (Merge pull request #430 from byte-physics/mb/feature/tracing-inside-im, 2023-03-09) Env: Windows 10, IP9 r39863

When executing Tracing.pxp locally I get

•run()
  Setting up tracing...
  Recompiling...
  Setting up tracing...
  Recompiling...
  Start of test "Unnamed"
  Entering test suite "Tracing.ipf"
  Entering test case "UTF_TestTracing#TracingTest"
  Leaving test case "UTF_TestTracing#TracingTest"
  Entering test case "UTF_TestTracing#TracingTest2"
  Assuming equality using mode WAVE_DATA for waves wv (0x2cd417d0) and _free_ (0x154c84d0); detailed: Waves differ in dimension sizes
Wave1: wv (0x2cd417d0)
Wave2: _free_ (0x154c84d0)
 Dimension Sizes |
-----------------|
 [56][2][0][0]   |
 [56][0][0][0]   |
: is false. Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
  Assuming equality using mode WAVE_SCALING for waves wv (0x2cd417d0) and _free_ (0x154c84d0): is false. Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
  Assuming equality using mode DIMENSION_UNITS for waves wv (0x2cd417d0) and _free_ (0x154c84d0): is false. Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
  Assuming equality using mode DIMENSION_LABELS for waves wv (0x2cd417d0) and _free_ (0x154c84d0); detailed: 
Non-empty dimension vs empty dimension: is false. Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
  Assuming equality using mode DIMENSION_SIZES for waves wv (0x2cd417d0) and _free_ (0x154c84d0): is false. Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
  Leaving test case "UTF_TestTracing#TracingTest2"
  Failed with 5 errors
  Leaving test suite "Tracing.ipf"
  Test finished with 5 errors
    ▶ Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
    ▶ Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
    ▶ Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
    ▶ Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
    ▶ Assertion "CHECK_EQUAL_WAVES(marker, markerRef)" failed in UTF_TestTracing#TracingTest2 (Tracing.ipf, line 78)
  End of test "Unnamed"
  Generating coverage output.
  .Done.
  Code lines: 30
Lines covered : 26
Coverage: 86.7%

@Garados007 Does that pass on your end? Why does CI then passes?

t-b commented 1 year ago

From https://gitlab.byte-physics.de:4001/mirror/igortest/-/jobs/80005

$ exit $(grep -e "errors=\"[0-9]\+\"" -e "failures=\"[0-9]\+\"" tests/**/JU_*.xml | grep -cv "failures=\"0\" errors=\"0\"")
grep: tests/**/JU_*.xml: No such file or directory
t-b commented 1 year ago

IMHO the bash shell does not understand **.

Garados007 commented 1 year ago

IMHO the bash shell does not understand **.

bash understands ** (source) and replaces it with the found files or directories. But if no files or directories are found it won't do something and grep print an error.

t-b commented 1 year ago

@Garados007

bash understands ** (source) and replaces it with the found files or directories. But if no files or directories are found it won't do something and grep print an error.

Thanks. Got it.

The failing job is now fixed but in https://gitlab.byte-physics.de:4001/mirror/igortest/-/jobs/81130 I still see

$ exit $(grep -e "errors=\"[0-9]\+\"" -e "failures=\"[0-9]\+\"" tests/**/JU_*.xml | grep -cv "failures=\"0\" errors=\"0\"")
grep: tests/**/JU_*.xml: No such file or directory
Garados007 commented 1 year ago

I fixed it in ~5ec8c9d (CI: fail execution if no JUnit file found, 2023-03-13).~ 4c4bd2e (CI: fail execution if no JUnit file found, 2023-03-13) (I forgot to add wc -l).