project-chip / certification-tool

A test harness and tooling designed to simplify development, testing, and certification for devices, guided by the Connectivity Standards Alliance.
https://csa-iot.org/
Apache License 2.0
41 stars 23 forks source link

[Bug] Test cases needs to be moved from Python Testing Suite - No commissioning to Python Testing Suite #410

Closed KishokG closed 1 month ago

KishokG commented 1 month ago

Describe the bug

Below mentioned tescases are listed under the Python Testing Suite - No commissioning in the TH UI. But this tescase needs to be commissioned before the test execution hence this testcase needs to be moved to the Python Testing Suite.

Steps to reproduce the behavior

  1. From the Test-Harness user interface loaded the PICS file to select the test cases.
  2. Selected the python test case in Python Testing Suite - No commissioning section to executed.
  3. Click the Start button on Test-Harness user interface.

Expected behavior

The testcase needs to be commissioned before the start python test.

Log files

UI_Test_Run_2024_09_09_16_24_55.log

PICS file

PICS.zip

Screenshots

image

Environment

Version: v2.11-beta3+fall2024 Sha: f52d40a

Additional Information

No response

fabiowmm commented 1 month ago

This is a script issue. Test scripts need to follow the example in connectedhomeip/src/python_testing/hello_test.py. Take a look at the comments in the test_TC_NAMES_2_1 method. It's just a matter of setting the is_commissioning variable.

jamesharrow commented 1 month ago

I've created an SDK PR with this change in - https://github.com/project-chip/connectedhomeip/pull/35560

Please review

jamesharrow commented 1 month ago

The PR for SDK has now merged which includes the is_commissioning=True https://github.com/project-chip/connectedhomeip/pull/35560

rquidute commented 1 month ago

Hi @jamesharrow The SDK SHA 17b1a38e909e7874593bcb87c31be03a5866f1d4 is the one picked for SVE, so if you want to verify the changes, you can validate by side loading the modified version.

KishokG commented 1 month ago

Issue fixed