search

tcunit / TcUnit

An unit testing framework for Beckhoff's TwinCAT 3
Other
273 stars 75 forks source link

Each message is printed multiple times in the XAE instance, and in every XAE instance #248

Closed TobiasKnauss closed 1 month ago

TobiasKnauss commented 1 month ago

Describe the bug Each message from TcUnit is printed many times in the Error List.

To Reproduce

And if you have multiple instances of XAE open, it will be printed in (almost) all of these instances, and in every instance as often as the number of projects in the solution that is loaded in this instance.

Expected behavior Each message should be printed only once. And only in the XAE instance where executed.

Screenshots grafik 20 projects in "Examples" solution: grafik 9 Failures and 20 Messages, times 20 projects: grafik 5 projects in solution "GNT Tc3 Libraries", 9 Failures and 20 Messages, times 5 projects (although no project of this solution is executed): grafik 3 projects in solution "GNT Tc3 Library Tests", ... like before... grafik

Software versions

Run environment Laptop

Additional context None that seems relevant.

sagatowski commented 1 month ago

Duplicate of #https://github.com/tcunit/TcUnit/issues/246 and https://github.com/tcunit/TcUnit/issues/92.

Is it possible you can update the TcUnit FAQ and create a PR to include this question? Thanks.

TobiasKnauss commented 1 month ago

Thanks for your quick reply.

For your comment in #92

I never considered the case of multiple TwinCAT projects in one solution. What is the use case for this? How does it differ from having multiple PLC projects in one TwinCAT project?

I can add that there are multiple reasons:

I will update the FAQ and create another PR.

TobiasKnauss commented 1 month ago

FAQ update is included in the first PR.

I have read the original issue: You might ask Beckhoff again if they are going to fix this, or maybe ask them to introduce a new function that sends to a single target only. And if they don't, you might want to look at https://infosys.beckhoff.com/english.php?content=../content/1033/tc3_eventlogger/index.html; maybe this has the functionality you need here.

sagatowski commented 1 month ago

I'll look at the PR.

It's not going to help asking Beckhoff again when they have clearly stated that this is not a problem but "according to design" (without giving any further information). What is going to help is if more people (other than me) state that this is a problem.

How is the event logger going to help?

sagatowski commented 1 month ago

Thanks for your quick reply.

For your comment in #92

I never considered the case of multiple TwinCAT projects in one solution. What is the use case for this? How does it differ from having multiple PLC projects in one TwinCAT project?

I can add that there are multiple reasons:

  • Multiple example projects in one solution, for testing with different I/O configuration or different CPU and task settings, or just because it feels "natural" to have on project per example.
  • If you use "Empty PLC projects" as suggested by Beckhoff for creating libraries (see my PR on doc update), you will have multiple library projects in a solution because you cannot have more than one PLC per "Empty PLC project".
  • Multiple open XAE windows shows this problem cross-instance.

I will update the FAQ and create another PR.

My point is that this is not an issue with TcUnit. If you create ADS-messages (with only the Beckhoff libraries, NOT using TcUnit) you will still get the same behaviour. This is something that is inherently built into TwinCAT.

I can't do anything about it. Beckhoff can.

TobiasKnauss commented 1 month ago

I can't do anything about it. Beckhoff can.

That's why I have asked them.

How is the event logger going to help?

I don't know, I haven't looked into that functionality yet. It was just an idea.

It looks like TcUnit (or Beckhoff) needs to find another way or fix the current functionality, because even with only one project in the solution and one XAE instance running, the messages are displayed out of order as you can see from the timestamps:

Message     04.10.2024 11:26:17 524 ms | 'PlcTask' (350): | Test class name=PRG_TEST.m_TestSuiteFb_FB_PulseCounter_to_PhysicalQuantityPerSecond             
Error       04.10.2024 11:26:17 644 ms | 'PlcTask' (350): | Failed tests: 1             
Error       04.10.2024 11:26:17 564 ms | 'PlcTask' (350): | ==========TESTS FINISHED RUNNING==========              
Error       04.10.2024 11:26:17 684 ms | 'PlcTask' (350): | ======================================              
Error       04.10.2024 11:26:17 604 ms | 'PlcTask' (350): | Tests: 4                
Error       04.10.2024 11:26:17 664 ms | 'PlcTask' (350): | Duration: 2.0004e-2             
Error       04.10.2024 11:26:17 584 ms | 'PlcTask' (350): | Test suites: 3              
Error       04.10.2024 11:26:17 624 ms | 'PlcTask' (350): | Successful tests: 3             
Message     04.10.2024 11:26:17 164 ms | 'PlcTask' (350): | Test suite ID=0 'PRG_TEST.m_TestSuiteFb_CreateFailureMessage'               
Message     04.10.2024 11:26:17 404 ms | 'PlcTask' (350): | Test name=Test_1                
Message     04.10.2024 11:26:17 484 ms | 'PlcTask' (350): | ID=2 number of tests=1, number of failed tests=0, duration=5.7e-6               
Message     04.10.2024 11:26:17 284 ms | 'PlcTask' (350): | Test assert type=ANY                
Message     04.10.2024 11:26:17 224 ms | 'PlcTask' (350): | Test class name=PRG_TEST.m_TestSuiteFb_CreateFailureMessage             
Message     04.10.2024 11:26:17 544 ms | 'PlcTask' (350): | Test status=SKIP, number of asserts=0, duration=0.0             
Message     04.10.2024 11:26:17 384 ms | 'PlcTask' (350): | ID=1 number of tests=1, number of failed tests=0, duration=5.4e-6               
Message     04.10.2024 11:26:17 184 ms | 'PlcTask' (350): | ID=0 number of tests=2, number of failed tests=1, duration=8.01e-5              
Message     04.10.2024 11:26:17 304 ms | 'PlcTask' (350): | Test name=Test_CreateFailureMessage_ValueOutOfRange_2               
Message     04.10.2024 11:26:17 364 ms | 'PlcTask' (350): | Test suite ID=1 'PRG_TEST.m_TestSuiteFb_FB_CompareToWarningFailureLimits'               
Message     04.10.2024 11:26:17 464 ms | 'PlcTask' (350): | Test suite ID=2 'PRG_TEST.m_TestSuiteFb_FB_PulseCounter_to_PhysicalQuantityPerSecond'               
Message     04.10.2024 11:26:17 204 ms | 'PlcTask' (350): | Test name=Test_CreateFailureMessage_ValueOutOfRange_1               
Message     04.10.2024 11:26:17 244 ms | 'PlcTask' (350): | Test status=FAIL, number of asserts=3, duration=6.43e-5             
Message     04.10.2024 11:26:17 344 ms | 'PlcTask' (350): | Test status=PASS, number of asserts=1, duration=7.5e-6              
Message     04.10.2024 11:26:17 444 ms | 'PlcTask' (350): | Test status=SKIP, number of asserts=0, duration=0.0             
Message     04.10.2024 11:26:17 324 ms | 'PlcTask' (350): | Test class name=PRG_TEST.m_TestSuiteFb_CreateFailureMessage             
Message     04.10.2024 11:26:17 424 ms | 'PlcTask' (350): | Test class name=PRG_TEST.m_TestSuiteFb_FB_CompareToWarningFailureLimits             
Message     04.10.2024 11:26:17 504 ms | 'PlcTask' (350): | Test name=Test_1
sagatowski commented 1 month ago

It's not a bug in TcUnit or TwinCAT.

Please read the introduction instruction manual for TcUnit:

https://tcunit.org/#/introduction-user-guide