[Bug] UI Error During Multiple Test Case Validation: Code 1011 - Keepalive Ping Timeout and Missing Close Frame in TV-APP

raju-apple commented 1 year ago

From chip-certification-tool created by Rajashreekalmane: CHIP-Specifications/chip-certification-tool#986

Describe the bug

If we run the UI test runner by selecting one or two test cases, there are no issues. However, when selecting more than five test cases or two test cases with a larger number of test steps, the test runner ends with the following error: 'Sent 1011 (unexpected error) keepalive ping timeout; no close frame received.

Steps to reproduce the behavior

From the Test-Harness user interface loaded the PICS xml file to select the test cases.
Selected all the Concentration measurement test case to executed.
Executed the below mentioned command to put DUT into a commissionable state, ./chip-tv-app
Click the Start button on Test-Harness user interface.

Log files

automated_2023_10_05_11_24_49.log

PICS file

Media Cluster Test Plan.txt

Screenshots

Environment

Version : TH Fall2023 Sha : 3f84bff5 DUT : tv-app

Additional Information

No response

raju-apple commented 1 year ago

I couldn't identify why this is happening, but I found that the error is happening in the _run function from TestRunner in runner.py. There's a big try:except: there, not sure where exactly the exception is being fired though.

raju-apple commented 1 year ago

There's an open issue in the SDK repo for this error: https://github.com/project-chip/connectedhomeip/issues/29194

fabiowmm commented 2 weeks ago

Deprioritized.

project-chip / certification-tool