search

tcunit / TcUnit

An unit testing framework for Beckhoff's TwinCAT 3
Other
271 stars 74 forks source link

Total number of asserts #157

Closed Alec-Mack closed 2 years ago

Alec-Mack commented 3 years ago

Hi @sagatowski

I'm really impressed with the TcUnit suite of tools. Thank you for creating them.

Would it be possible to return the total number of asserts in the test summary please? (The Error ADS Log table you create) Or is there an option for this that I haven't found?

Many thanks

Alec

sagatowski commented 3 years ago

Hi @Alec-Mack! Is there any particular reason why you would want to display this data?

The data is available as a variable. You can check the variable AssertResultInstances in every instance of AssertResults of every test suite (FB_TestSuite). Check the description of FB_AssertArrayResultStatic. I think this should give you the information you need.

Alec-Mack commented 3 years ago

Hi Jakob

I would like to display it as it is a big number and I think it would give me a warm feeling seeing it grow as I write more tests.

Alec

From: Jakob Sagatowski @.> Sent: 22 July 2021 08:36 To: tcunit/TcUnit @.> Cc: Mack, Alec @.>; Mention @.> Subject: CAUTION: External email - Re: [tcunit/TcUnit] Total number of asserts (#157)

Hi @Alec-Mackhttps://github.com/Alec-Mack! Is there any particular reason why you would want to display this data?

The data is available as a variable. You can check the variable AssertResultInstances in every instance of AssertResults of every test suite (FB_TestSuite). Check the description of FB_AssertArrayResultStatic. I think this should give you the information you need.

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/tcunit/TcUnit/issues/157#issuecomment-884713138, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU5CKW3LPOWC62UIBFUM233TY7C53ANCNFSM5AV4NEDQ.

sagatowski commented 3 years ago

Hi Alec.

For this reason it's enough that you see the amount of succesful and failed tests. You should anyway strive to have only one (or few) assert per test. Please see:

https://softwareengineering.stackexchange.com/questions/7823/is-it-ok-to-have-multiple-asserts-in-a-single-unit-test

Alec-Mack commented 3 years ago

Thanks Jakob

I’m new to Unit testing so I’m not familiar with some of the philosophies. Every day is a learning day.

Alec

From: Jakob Sagatowski @.> Sent: 22 July 2021 10:27 To: tcunit/TcUnit @.> Cc: Mack, Alec @.>; Mention @.> Subject: CAUTION: External email - Re: [tcunit/TcUnit] Total number of asserts (#157)

Hi Alec.

For this reason it's enough that you see the amount of succesful and failed tests. You should anyway strive to have only one (or few) assert per test. Please see:

https://softwareengineering.stackexchange.com/questions/7823/is-it-ok-to-have-multiple-asserts-in-a-single-unit-test

— You are receiving this because you were mentioned. Reply to this email directly, view it on GitHubhttps://github.com/tcunit/TcUnit/issues/157#issuecomment-884774640, or unsubscribehttps://github.com/notifications/unsubscribe-auth/AU5CKW55DMJ4AJYSVODYOLLTY7P4TANCNFSM5AV4NEDQ.

sagatowski commented 3 years ago

Hey Alec!

No problem, everyone is new to everything at some point :)

If you think this answer is OK, can you close the issue? Thanks.

/ Jakob

Alec-Mack commented 3 years ago

Hello Jakob I hope you are well. I've run into the problem that Arkonos reported here: https://github.com/tcunit/TcUnit/issues/134 where many asserts caused a PageFile issue and BSODs. I've increased the 3 TcUnit MaxNumberOf... parameters, and while the tests now pass, I still get a BSOD. I have many asserts which make very repetitive checks and I am going through my tests to make sure that I don't do asserts on these by doing this sort of thing:

IF NOT (Result = LastResult AND Expected = LastExpected) THEN
        AssertEquals(Expected := Expected,
                 Actual := Result,
                 Message := 'Display fault message.');
END_IF

Which seems to be doing the trick and reducing the number of asserts. Now I would like to know the total assert count, not for a warm fuzzy feeling, but to make sure that I don't approach conditions that might BSOD my development machine.

From reading your documentation for FB_AssertResultStatic it looks like I shouldn't need to do the above. Maybe something isn't quite right in there or maybe I've misunderstood? I know Unit testing philosophy says the number of asserts should be low, but I have conditions in the machine the PLC will eventually control that need to persist for reasonable (in human terms) amounts of time.

To collect the number of asserts, I tried following your advice, but AssertResults doesn't seem to be a method of my FB_TestSuite.

FB_Test.GetNumberOfAssertions() looks like it could help, but it is INTERNAL and is keeping its information to itself :-).

Are you able to provide any further advice please?

Many thanks

Alec

sagatowski commented 2 years ago

Hi! Sorry for very late reply. What type of PLC are you running the tests on? How much AMS/Router memory is allocated to the runtime? Might it simply be a memory problem? That's the only reason I can think of that you get BSOD.

sagatowski commented 2 years ago

@Alec-Mack, have you checked this issue? https://github.com/tcunit/TcUnit/issues/148

Alec-Mack commented 2 years ago

Hi @sagatowski It might have been a memory problem. I left the 'If nothing has changed, don't bother doing the assert' check and everything behaved itself. I've moved onto other things, so I'll close the issue. Thanks for the support. Alec