[TC-S-2.3] Test case is failing due to the group messaging is not functioning in the UI, causing the group cast method-related test steps to fail.

[Ashwinigrl]

Description:

• [ ] TC-S-2.3

Step 6d states that TH sends a RemoveScene command to group G1, specifying the GroupID field as G1 and the SceneID field as 0x03. However, the Scene Id 0x03 is not removed as expected. Subsequently, the verification steps involving View Scene and GetSceneMembership commands, These steps are failing due to discrepancies in the Status value, SceneList value, and Capacity value.

The RemoveScene command is being sent using the group cast method. It appears that group messaging is not functioning correctly in the UI, resulting in the RemoveScene command failing to remove the scene value. Consequently, the dependent test steps are also failing.

• Dependent Test steps 6e, 7d, 8b, 8d, and 9b, 10a are failing in the UI due to the discrepancies in the Status value and Capacity value.

Group cast method related test steps: 5b, 6a, 6c, 6d, 6f, 6h, 8c, 9a, and 10a.

Test steps 5b, 6c: These steps have no dependencies, so no other steps are failing because of them.

Test step 6a: This is a user prompt step. When the command is run manually, group messaging works; hence, the dependent test step 6b is not failing.

Test steps 6d, 6h: The 'Remove' command does not remove the scene value with the group cast method, resulting in the dependent steps 6e and 6i failing. and effecting the test steps 7d, 8b, 8d also.

Test step 6f: This step involves sending the 'Store Scene' command using the group cast method. The test step and its dependent step 6g are passing, this is a false pass, as the scene value is not removed in step 6d, causing the same value to appear in step 6g. Therefore, it is unclear if the store command works correctly with the multicast method.

Test steps 7d, 8b, 8d, and 9b: These are failing due to a mismatch in the scene list IDs. Since the scene values are neither removed nor copied as expected, the list values are incorrect, resulting in test failures.

No issues were encountered during manual validation by command execution.(Facing issue in UI validation only)

Steps to reproduce:

Test steps 6b, 6e, 7d, 8b, 8d, 9b, 10a

Reference Logs:

https://github.com/user-attachments/files/17501637/TC-S-2.3.UI.Log.txt

Screen shot:

Reference Logs:

TH logs: https://github.com/user-attachments/files/17501637/TC-S-2.3.UI.Log.txt

UI Prompt logs: https://github.com/user-attachments/files/17501643/TC-S-2.3.User.prompt.log.txt

PICS File used: https://github.com/user-attachments/files/17501647/Scenes.Management.Cluster.Test.Plan.zip

Additional Info:

TH Version: v2.11+fall2024 Sha: 8f86ccd

[raul-marquez-csa]

PR https://github.com/project-chip/connectedhomeip/pull/32625

[Ashwinigrl]

@raul-marquez-csa Still facing the failure in Test step 6b, 6e, 7d, 8b, 8d, 9b, 10a.

TH Version: 2.10+spring2024 Sha: 5f28838

Here is the Screenshot:

TH logs: TC-S-2.3. txt

UI Prompt logs: TC-S-2.3(UI Prompt ) log.txt

PICS File used: Scenes Management Cluster Test Plan.zip

[raul-marquez-csa]

@Ashwinigrl, can you please verify if this is still a problem?

[Ashwinigrl]

@raul-marquez-csa Validated test case in latest TH Image Version: v2.11+fall2024 Sha: 8f86ccd. Still facing issue in same test steps.

Here is the Screenshot:

TH logs: TC-S-2.3(UI Log).txt

UI Prompt logs: TC-S-2.3(User prompt log).txt

PICS File used: Scenes Management Cluster Test Plan.zip

[raul-marquez-csa]

@Ashwinigrl, from your screenshot

In between steps 6d and 6e, a 1+ second delay is expected "to give CI time to process the RemoveScene command" This is the YAML code for that particular delay

    - label: "Wait 1+ s to give CI time to process the RemoveScene command."
      PICS: PICS_SDK_CI_ONLY
      cluster: "DelayCommands"
      command: "WaitForMs"
      arguments:
          values:
              - name: "ms"
                value: 1250

Some questions (sorry if some too obvious):

• Judging from the label of the delay step, and from the PICS_SDK_CI_ONLY line, does the delay only occur in the CI?
• The symbol next to the delay step is for skipped steps, does running the test in the TH/raspi not require the delay?
• Ultimately, might the problem be that after executing the remove scene command (6d), no delay is given for it to complete, and thus the view scene command (6e) STILL finds it in the table (returning SUCCESS instead of the expected NOT_FOUND status code)?

Most failed steps in your screenshot happen after delay steps which appear to be skipped.

[lpbeliveau-silabs]

@Ashwinigrl @raul-marquez-csa Step 6d is a groups command. Group messaging is not working in UI. It seems this issue is not a Scenes specific problem. Do we have an issue about group commands in UI?

[raul-marquez-csa]

Taking a closer look, all the failed steps a related to read commands (ViewScene, GetSceneMembership), so I still suspect it has something to do with waiting it out.

Step 6d is a groups command. Group messaging is not working in UI. It seems this issue is not a Scenes specific problem. Do we have an issue about group commands in UI?

@lpbeliveau-silabs, I was unable to find an issue about group commands in UI, are you able to open one so we can see in detail what the issue is?

@Ashwinigrl, are you familiar with this?