search

project-chip / connectedhomeip

Matter (formerly Project CHIP) creates more connections between more objects, simplifying development for manufacturers and increasing compatibility for consumers, guided by the Connectivity Standards Alliance.
https://buildwithmatter.com
Apache License 2.0
7.43k stars 1.99k forks source link

[BUG] [TC-ICDM-2.1] test execution calls 2 hook methods for the same step - Test Harness #35015

Open rquidute opened 2 months ago

rquidute commented 2 months ago

Reproduction steps

Using Test Harness Run TC-ICDM-2.1 test case At step TH reads from the DUT the UserActiveModeTriggerHint attribute, TH is getting notified by 2 hook methods (step_skipped and then step_failure)

Expected Result: The test should skip the step TH reads from the DUT the UserActiveModeTriggerHint attribute and then try to execute the next step.

TH Screenshot: Screenshot 2024-08-15 at 11 14 35

TH Execution log: UI_Test_Run_2024_08_15_13_06_27.log

Bug prevalence

Always

GitHub hash of the SDK that was being used

d0d91272068f267cf880f9d56787ca28da885673

Platform

python

Platform Version(s)

No response

Anything else?

No response

rquidute commented 2 months ago

Hi @cecille, just tagging you here in case you have any thoughts about this. Thanks in advance.

tcarmelveilleux commented 2 months ago

Wrong sequences of hooks/matter_testing_support call that are detectable as completely wrong should cause the test to end in ERROR not FAIL or PASS or hang.

github-actions[bot] commented 1 month ago

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Remove stale label or comment or this will be closed in 30 days.