search

project-chip / certification-tool

A test harness and tooling designed to simplify development, testing, and certification for devices, guided by the Connectivity Standards Alliance.
https://csa-iot.org/
Apache License 2.0
40 stars 23 forks source link

[ICDM] UI-automation commissioning is failing on the RPI platform using lit-icd-app. #153

Closed Rajashreekalmane closed 2 months ago

Rajashreekalmane commented 8 months ago

Describe the bug

[ICDM] UI-automation commissioning is failing on the RPI platform using lit-icd-app.

image

Steps to reproduce the behavior

  1. Open the UI
  2. Upload the PICS
  3. Ensure the TC-ICDM-1.1 test case is selected
  4. Executed the below mentioned command to put DUT into a commissionable state,
  5. ./lit-icd-app
  6. Click the Start button on Test-Harness user interface.

Version: 2.10-beta2.1+spring2024 Sha: 549e33f

ccruzagralopes commented 8 months ago

Hello @Rajashreekalmane, can you please provide the test case logs?

forrestpangpang commented 8 months ago

UI_Test_Run_2024_01_31_09_12_05.log I got the same issue with NXP LIT ICD device. Logs attached. lit-icd-issue

ccruzagralopes commented 8 months ago

From the logs, the error was Buffer too small:

CHIPTOOL   | 2024-01-31 09:12:39.737054 | CHIP:TOO Failed to persist symmetric key for E6E3853A0978F865: src/lib/core/TLVWriter.cpp:846: CHIP Error 0x00000019: Buffer too small
CHIPTOOL   | 2024-01-31 09:12:39.738022 | CHIP:TOO Run command failure: src/lib/core/TLVWriter.cpp:846: CHIP Error 0x00000019: Buffer too small

This seems like an error in the pairing between the DUT and the chip-tool binary, not related to the TH. @cecille can you help with this issue?

cecille commented 8 months ago

This might actually be a TH problem - there's only so much space on the commissioner to store keys. If you've commissioned a bunch of devices, you'll run out of space.

Does this error occur if the lit-icd app is the FIRST one you test?

Rajashreekalmane commented 8 months ago

@cecille we have commissioned only lit-icd-app , even though facing the issue after clearing the tmp files

ccruzagralopes commented 8 months ago

This might actually be a TH problem - there's only so much space on the commissioner to store keys. If you've commissioned a bunch of devices, you'll run out of space.

@cecille TH destroys the SDK container after each test suite execution, so each time you execute the test cases you get a "clean" container, without previously stored keys.

dvdm-qorvo commented 4 months ago

We are hitting the same issue with our LIT ICD device... It looks like chip-tool it is not able to store the combination of the big nodeId number (generated by the harness) and the symmetric key. If you executed chiptool with a small nodeId number (1 for example), this error can't be reproduced

antonio-amjr commented 4 months ago

Hi @Rajashreekalmane and @dvdm-qorvo

It seems that your theory @dvdm-qorvo about the nodeID number size proceeds. I did reproduce the behavior with the 64 random bits number and I also managed to commit successfully with a smaller nodeID. Looking at the code, I suppose the problem is with the MaxICDClientInfoSize() method from the SDK's DefaultICDClientStorage.h file is not correct.

But the good news is that the a fix is already merged in the SDK's code (from a commit some time ago) and the fall2024 beta version of TH is already using a SDK with the relevant change for this.

I already tested here and everything seems fine. Is it possible for you guys @dvdm-qorvo and @Rajashreekalmane test as well the v2.11-beta2+fall2024 branch and verify if it solves this issue?

Let me know any news, Thanks

dvdm-qorvo commented 4 months ago

@antonio-amjr , Thanks for your reply. I can confirm with a chip-tool build that has this commit onboard, that it works. The problem is that we are running tests against Matter v1.3 TH which has a chip-tool that does not contain this fix. Can we simply update the TH with the fix and share the update patch with ATL to get things working? Or a formal patch release need to be made of the TH to have a valid certification test round?

antonio-amjr commented 4 months ago

@dvdm-qorvo,

Since you're pursuing a certification round of tests, I believe a formal patch release will be necessary. But I'm not the best person to answer that. What I recommend is for you to ask this to someone in the certification team. I think the CSA Slack channel general-matter-wg is a good start point.

PS: Thanks for the feedback

antonio-amjr commented 2 months ago

I believe we don't need further investigation on this and the current fall2024 seems to already solve this issue. Since the only questions left is related to the certification process, I'm closing this up.