search

tcunit / TcUnit

An unit testing framework for Beckhoff's TwinCAT 3
Other
258 stars 72 forks source link

Feature/add test timeout #181

Open Navid-Mashayekh opened 2 years ago

Navid-Mashayekh commented 2 years ago

New function "TIMEOUT_FOR_EACH_TEST" added to control time out for each test. This implements the functionality for https://github.com/tcunit/TcUnit/issues/142.

sagatowski commented 2 years ago

I'll start reviewing this PR as soon as this PR is completed/merged. Once it's merged, you will most likely need to merge from trunk again as the changes in the PR #159 will most likely break your branch.

I realized that there might be quite a few of these annotations the more functions we want to add to TcUnit. Is there any way we could achieve annotations in the same way as JUnit?

What does this PR do compared to PR #179? What do you want to happen if you mix AWAIT_ASSERTIONS and TEST_TIMEOUT?

What happens if a timeout happens?

Could you also link this to an issue (that describes what problem this solves or what feature is added? As it is right now I have to read the code to maybe understand what problem this solves).

sagatowski commented 2 years ago

I just relized that this is connected to https://github.com/tcunit/TcUnit/issues/142.

sagatowski commented 1 year ago

@Navid-Mashayekh How does this differ from #179? When would one want to use one over the other? Aren't they redundant?

The standard procedure is to simply use a timer or a state-machine to control the timeout for a test.

Navid-Mashayekh commented 1 year ago

The main idea behind the timeout is to be able to quit the test in a pipeline when it stucks in a test (eq: missing TEST_FINISHED, etc) otherwise the job will keep the ci/cd busy for hours!

179 is about delaying the assertions, because unlike pc software many plc logics are based on cyclic processing. In these cases as you said we should use a timer in the test suit or a state machine. I think it might be useful to have this feature integrated, but having both (timeout and delay) is also tricky! since they are in contrast they can break tests unintentionaly!

sagatowski commented 9 months ago

I see! This is redundant to the timeout flag in the TcUnit-Runner.

-u [OPTIONAL] Timeout the process(es) with an error after X minutes

The build job will anyway continue to run unless there is an actual return from the process of which it invoked.