search

project-chip / certification-tool

A test harness and tooling designed to simplify development, testing, and certification for devices, guided by the Connectivity Standards Alliance.
https://csa-iot.org/
Apache License 2.0
38 stars 23 forks source link

Test_TC_MEDIAPLAYBACK_6_8 Test case - The test case is "Semiautomated," as per Repo but it is validated as "Fully Automated" in the image #189

Open Rajashreekalmane opened 7 months ago

Rajashreekalmane commented 7 months ago

Describe the bug

Test_TC_MEDIAPLAYBACK_6_8.yaml is marked as "Semiautomated" in Repo. However, when validating the test case in the image provided, it is evident that the test case is executed as "Fully Automated."

Steps to reproduce the behavior

  1. Open the UI
  2. Upload the PICS
  3. Ensure the TC_MEDIAPLAYBACK_6_8 test case is selected
  4. Executed the below mentioned command to put DUT into a commissionable state, ./chip-tv-app
  5. Click the Start button on Test-Harness user interface.

Expected behavior

No response

Log files

UI_Test_Run_2024_02_22_11_12_35.log

PICS file

Media Cluster Test Plan.txt

Screenshots

image

Environment

Version: 2.10-beta3+spring2024 Sha: 1fd1429

Additional Information

YMAL Script Reference : https://github.com/project-chip/connectedhomeip/blob/1.3-sve/src/app/tests/suites/certification/Test_TC_MEDIAPLAYBACK_6_8.yaml

rquidute commented 7 months ago

Hi @Rajashreekalmane I can't see in YAML script file (TC-MEDIAPLAYBACK-6.8.yaml) any step that requires User Prompt, this is how TH marks the test case as Semi-automated. @manjunath-grl Could you please double check if the YAML script file for TC-MEDIAPLAYBACK-6.8 test case?

In addition, the Steps to reproduce the behavior in this issue, it seems to be related to a different issue, right?

rquidute commented 7 months ago

Hi @raul-marquez-csa / @gvargas-csa Could you please double check if the YAML script file for TC-MEDIAPLAYBACK-6.8 test case?