Add compilation test feature

[Garados007]

A set of new assertions are introduced that allow to test if specific Igor procedure files can be compiled. These assertions require to be used inside an independent module.

Close #6

[t-b]

@Garados007 I've chatted with Michael, he'll have a look, likely on tuesday, and get together with you about how to fix that.

[t-b]

When I execute the IMUnitTests.pxp experiment with IP from the docker image on debian I don't get a crash.

I did:

~/devel/igortest$ docker run -it -v /tmp/.X11-unix:/tmp/.X11-unix:rw -e DISPLAY=$DISPLAY -v $(pwd):/igortest registry.byte-physics.de:4001/internal/docker-igorpro:nightly
igor@842c2f43fda7:~$ ln -s /igortest/{procedures,tests} WaveMetrics/Igor\ Pro\ 9\ User\ Files/User\ Procedures/
igor@842c2f43fda7:~$ touch /igortest/tests/IMUnitTests/DO_AUTORUN.txt
igor@842c2f43fda7:~$ igorpro execute --verbose /igortest/tests/IMUnitTests/IMUnitTests.pxp 

Here is the stdout:

Start of test "IM Unit Tests"
Entering test suite "Test-CompilationTests [IM_TEST]"
Entering test case "TEST_Test_Compilation#CheckValidFile"
Exec: INSERTINCLUDE "TestData-Valid"
Exec: variable/G root:Packages:igortest:compilationState = 0x2
Exec: COMPILEPROCEDURES 
Exec: IM_TEST#SetCompilationFlag()
Exec: DELETEINCLUDE "TestData-Valid"
Exec: COMPILEPROCEDURES 
Exec: IM_TEST#RunTest(":COMPILATION_REENTRY:")
Leaving test case "TEST_Test_Compilation#CheckValidFile"
Entering test case "TEST_Test_Compilation#CheckInvalidFile"
Exec: INSERTINCLUDE "TestData-Invalid"
Exec: variable/G root:Packages:igortest:compilationState = 0x2
Exec: COMPILEPROCEDURES 
Exec: IM_TEST#SetCompilationFlag()
Exec: DELETEINCLUDE "TestData-Invalid"
Exec: COMPILEPROCEDURES 
Exec: IM_TEST#RunTest(":COMPILATION_REENTRY:")

Some observations:

• There is no JUNIT XML file generated, but I would have expected that

[Garados007]

Here is the stdout:

Is this the full stdout? I expect End of test "IM Unit Tests" at the end of the log. Only then it's not crashed.

[Garados007]

The Exec: ... is just a temporary debug log output to show what is enqueued to the operation queue. This is no indicator that it was successfully executed.

[t-b]

@Garados007 Yes that was the full stdout output. Here are also the logs.tar.gz from another run.

[Garados007]

Your results are completely different from what we have experienced from CI yesterday. It's crashing at different locations.

[Garados007]

But one thing stays the same. Igor has critical problems when ProgGlobal is in an uncompiled state because of invalid procedure files and a compiled independent module want's to do something with the operation queue. But this issue is only inside the CI. We couldn't reproduce this with different Igor versions locally.

[t-b]

I forgot to add the TUF XOP...

cp /igortest/xop/TUFXOP-64.xop Igor\ Extensions\ \(64-bit\)/

This does get me a bit further but still no XML file.

[t-b]

I do see an "early return" when executing

• igorpro --use-xserver execute --verbose /igortest/tests/IMUnitTests/IMUnitTests.pxp
• run() # inside IP

without DO_AUTORUN.txt.

[MichaelHuth]

In the macosx CI job I see an "10934 Abort trap: 6" when Igor64 is run. Though, I am not sure if there is still some issue with the CI setup there.

[t-b]

In the macosx CI job I see an "10934 Abort trap: 6" when Igor64 is run. Though, I am not sure if there is still some issue with the CI setup there.

Which is expected if you use DYLD_INSERT_LIBRARIES as that can only be used with a debug xop which was compiled with sanitizer support.

[t-b]

@MichaelHuth Can't find the igortest include file.

[t-b]

@MichaelHuth You create bogus links: iutf -> procedures tests -> tests

[t-b]

@MichaelHuth testing_macosx from the tuf xop does it properly.

[t-b]

@MichaelHuth There was a popup with the compilation error. I've cancelled that.

[MichaelHuth]

The popup issue is probably also present if the INSERTINCLUDE file does not exist? If so I guess the code should check for the existence.

[Garados007]

Am I seeing right? It seems to be working with MacOSX...

[Garados007]

Hmm. I need to fix the report generation for MacOSX but after all it's working.

[Garados007]

@t-b @MichaelHuth Ready for review. I fixed the report generation for MacOSX and uploaded the results to the staging area.

[Garados007]

During the fix of https://github.com/byte-physics/igortest/issues/432#issuecomment-1465883367 I found some bugs. I'm taking this PR back until it's done.

[Garados007]

@t-b @MichaelHuth Fixed it.

[t-b]

Review:

Whitespace issue:

$ git diff --check main..
procedures/igortest-basics.ipf:1613: indent with spaces.
+        // check if the reentry call originates from our own background monitor or compilation tester

1d18cb4d (Add ci job under macosx, 2023-03-07)

• MACOSX_XCODE_VERSION is unused
• GIT_SUBMODULE_STRATEGY should not be globally set but only for the MACOSX job. I also think that normal should be enough, see https://docs.gitlab.com/ee/ci/runners/configure_runners.html#git-submodule-strategy.

195a7add (Add TUF-64 XOP for MacOSX in xop folder, 2023-03-09)

Good.

c19a93d4 (CI: Add verbose output for Igor nightly, 2023-03-11)

Jeep.

91a737eb (Fix: Tracing tests, 2023-03-11)

Aha.

f946ff4b (Fix: Wave tracking and tracing in UnitTests for Igor <9, 2023-03-11)

Jeep.

51e42350 (Add compilation test feature, 2023-03-02)

Works nicely! One minor nit: It would be nice if Execute/P uses /Q as well so that the history is not cluttered.

c60bc857 (CI: Remove IMUnitTests from Linux CI, 2023-03-11) 4c4bd2e5 (CI: fail execution if no JUnit file found, 2023-03-13) 717165df (Fix: Function signature check in GetFunctionSignatureTCMD for Igor 6, 2023-03-13)

Makes sense. Thanks for having 51e42350 in its own commit with explanations.

[t-b]

@Garados007 The new way of testing compilation is nice! See https://github.com/AllenInstitute/MIES/blob/feature/1649-switch-to-IUTF-compilation-testing/Packages/tests/Compilation/CompilationTester.ipf how this is done now for MIES.

[Garados007]

@t-b Fixed it.