project-chip / certification-tool

A test harness and tooling designed to simplify development, testing, and certification for devices, guided by the Connectivity Standards Alliance.
https://csa-iot.org/
Apache License 2.0
27 stars 13 forks source link

[Bug] TC-CNET-4.16 semi-automated procedure needs to be updated #231

Open dkaytuv opened 3 months ago

dkaytuv commented 3 months ago

Describe the bug

When running TC-CNET-4.16 there are precondition instruction popups to commission the TH to the DUT however the test case then proceeds to attempt automated commissioning. If tester follows the prompts, test will fail to run to completion.

To workaround existing UI instructions follow prompt to reset the DUT to factory defaults and confirm by selecting "Pass." For subsequent instructions do not follow the prompt and select "Pass." Test will complete with automation.

Steps to reproduce the behavior

  1. Execute TC-CNET-4.16 following prompts to commission the DUT.

Expected behavior

  1. Test case pop-up prompts for manual instructions will only appear with commands the Tester must execute.

Log files

CNET-4.16_procedure.log

PICS file

No response

Screenshots

No response

Environment

TH: v2.10-beta3+spring2024

Additional Information

No response

dkaytuv commented 3 months ago

Also observed in:

CNET-4 6_prompt TC-CNET-4.6.log