project-chip / certification-tool

A test harness and tooling designed to simplify development, testing, and certification for devices, guided by the Connectivity Standards Alliance.
https://csa-iot.org/
Apache License 2.0
36 stars 22 forks source link

[Bug] TC-SC-4.1 check the discriminator not correct #144

Closed lxj199722 closed 3 months ago

lxj199722 commented 8 months ago

Describe the bug

Hello, For TC-SC-4.1, Step 4d and 4e, it requires to check the discriminator, and on default TH only recognises 3840. lf discriminator is changed, what should be set in the TH? My DUT is able to get commissioned, but when checking discriminator in Step 4d and e of TC-SC-4.1, it failed. TH Version: TH Fall2023, Sha: 3f84bff5.

Steps to reproduce the behavior

No response

Expected behavior

No response

Log files

No response

PICS file

No response

Screenshots

No response

Environment

No response

Additional Information

No response

rquidute commented 8 months ago

Hi @lxj199722 could you please attach log files and PICS file used in this scenario?

lxj199722 commented 8 months ago

Hi @lxj199722 could you please attach log files and PICS file used in this scenario? Hi, this is dut_config information and TC-SC-4.1 log files. image UI_Test_Run_TC_SC_41_2024_01_03_17_37_16.log

rquidute commented 8 months ago

Hi @cecille Do you have any idea what is happening?

cecille commented 8 months ago

This is not a problem with the TH using an incorrect value - the value appears in the test as expected, but it looks like maybe the discovery commands code is wrong.

CHIPTOOL   | 2024-01-03 09:39:20.117672 | CHIP:DIS Discovered node:
CHIPTOOL   | 2024-01-03 09:39:20.118590 | CHIP:DIS  Hostname: 34155B5D6D20
CHIPTOOL   | 2024-01-03 09:39:20.119490 | CHIP:DIS  IP Address #1: fe80::3629:8fff:fe1f:4215
CHIPTOOL   | 2024-01-03 09:39:20.120449 | CHIP:DIS  Port: 5540
CHIPTOOL   | 2024-01-03 09:39:20.121358 | CHIP:DIS  Mrp Interval idle: not present
CHIPTOOL   | 2024-01-03 09:39:20.122253 | CHIP:DIS  Mrp Interval active: not present
CHIPTOOL   | 2024-01-03 09:39:20.123140 | CHIP:DIS  TCP Supported: 1
CHIPTOOL   | 2024-01-03 09:39:20.124059 | CHIP:DIS  Rotating ID: 0200F666BFA30A0701DA7E8C619E396AC578
CHIPTOOL   | 2024-01-03 09:39:20.124952 | CHIP:DIS  Vendor ID: 4933
CHIPTOOL   | 2024-01-03 09:39:20.126646 | CHIP:DIS  Product ID: 16897
CHIPTOOL   | 2024-01-03 09:39:20.128575 | CHIP:DIS  Device Type: 65535
CHIPTOOL   | 2024-01-03 09:39:20.130858 | CHIP:DIS  Long Discriminator: 1300
CHIPTOOL   | 2024-01-03 09:39:20.131856 | CHIP:DIS  Pairing Hint: 36
CHIPTOOL   | 2024-01-03 09:39:20.132764 | CHIP:DIS  Instance Name: 8E90FB880A9C93B2
CHIPTOOL   | 2024-01-03 09:39:20.134008 | CHIP:DIS  Commissioning Mode: 1
CHIPTOOL   | 2024-01-03 09:39:20.134941 | CHIP:DIS Discovered node:
CHIPTOOL   | 2024-01-03 09:39:20.135856 | CHIP:DIS  Hostname: 34155B5D6D20
CHIPTOOL   | 2024-01-03 09:39:20.136759 | CHIP:DIS  IP Address #1: fe80::3629:8fff:fe1f:4215
CHIPTOOL   | 2024-01-03 09:39:20.137634 | CHIP:DIS  Port: 5540
CHIPTOOL   | 2024-01-03 09:39:20.138497 | CHIP:DIS  Mrp Interval idle: not present
CHIPTOOL   | 2024-01-03 09:39:20.139344 | CHIP:DIS  Mrp Interval active: not present
CHIPTOOL   | 2024-01-03 09:39:20.140224 | CHIP:DIS  TCP Supported: 1
CHIPTOOL   | 2024-01-03 09:39:20.141079 | CHIP:DIS  Rotating ID: 0200F666BFA30A0701DA7E8C619E396AC578
CHIPTOOL   | 2024-01-03 09:39:20.141915 | CHIP:DIS  Vendor ID: 4933
CHIPTOOL   | 2024-01-03 09:39:20.142750 | CHIP:DIS  Product ID: 16897
CHIPTOOL   | 2024-01-03 09:39:20.143586 | CHIP:DIS  Device Type: 65535
CHIPTOOL   | 2024-01-03 09:39:20.144478 | CHIP:DIS  Long Discriminator: 1300
CHIPTOOL   | 2024-01-03 09:39:20.145323 | CHIP:DIS  Pairing Hint: 36
CHIPTOOL   | 2024-01-03 09:39:20.146152 | CHIP:DIS  Instance Name: 8E90FB880A9C93B2
CHIPTOOL   | 2024-01-03 09:39:20.146980 | CHIP:DIS  Commissioning Mode: 1
CHIPTOOL   | 2024-01-03 09:39:20.147857 | CHIP:CTL Unknown filter type; all matches will fail
WARNING    | 2024-01-03 09:39:20.149414 | Test Failure: The test expectation "longDiscriminator == 1300" is false
INFO       | 2024-01-03 09:39:20.153149 | Test Step Completed [FAILED]: Step 2h: key D must be present and represents the discriminator which must be encoded as a variable-length decimal value with up to 4 digits omitting any leading zeros

^ longDiscriminator == 1300 indicates that the script substituted the value correctly for the check, but longDiscriminator is incorrect, despite being obviously correct.

cecille commented 8 months ago

I wonder if there are some weird characters in there or something. The logs indicate the discovered device is correct, and I THINK they're pulling that from the TXT. It would be good to get an additional source of truth on the TXT record. Can you run avahi-browse -r _matterc._udp while the device is up on the network and send the results?

cecille commented 8 months ago

This also isn't the first failure - this step fails earlier:

CHIPTOOL   | 2024-01-03 09:39:17.452486 | CHIP:TOO Command: discover find-commissionable-by-short-discriminator 15 
CHIPTOOL   | 2024-01-03 09:39:17.455221 | CHIP:DL Avahi browse: cache exhausted
CHIPTOOL   | 2024-01-03 09:39:17.456783 | CHIP:DL Avahi browse: all for now
CHIPTOOL   | 2024-01-03 09:39:17.457705 | CHIP:DL Avahi browse: cache exhausted
CHIPTOOL   | 2024-01-03 09:39:17.458576 | CHIP:DL Avahi browse: all for now
CHIPTOOL   | 2024-01-03 09:39:17.459436 | CHIP:DL Avahi browse: cache exhausted
CHIPTOOL   | 2024-01-03 09:39:17.460337 | CHIP:DL Avahi browse: all for now
CHIPTOOL   | 2024-01-03 09:39:17.461213 | CHIP:DL Avahi browse: cache exhausted
CHIPTOOL   | 2024-01-03 09:39:17.462077 | CHIP:DL Avahi browse: all for now
CHIPTOOL   | 2024-01-03 09:39:17.462929 | CHIP:DL Avahi browse: cache exhausted
CHIPTOOL   | 2024-01-03 09:39:17.463786 | CHIP:DL Avahi browse: all for now
WARNING    | 2024-01-03 09:39:17.465437 | Test Failure: The test expects no error but the "FAILURE" error occured.
INFO       | 2024-01-03 09:39:17.468608 | Test Step Completed [FAILED]: Step 2d: Check Short Discriminator (_S)
cecille commented 8 months ago

So if you could also run a check for avahi-browse _services._dns-sd._udp, that would narrow down the first failure.

KyuminAhn commented 1 month ago

I am experiencing the same problem. Is there a solution?