search

project-chip / connectedhomeip

Matter (formerly Project CHIP) creates more connections between more objects, simplifying development for manufacturers and increasing compatibility for consumers, guided by the Connectivity Standards Alliance.
https://buildwithmatter.com
Apache License 2.0
7.18k stars 1.9k forks source link

[CERT-TEST-FAILURE] [TC-IDM-10.1] Test Failure #30547

Open Saravana-kr22 opened 7 months ago

Saravana-kr22 commented 7 months ago

Feature Area

Other

Test Case

TC-IDM-10.1

Reproduction steps

Command to execute the IDM-10.1 : python3 TC_DeviceBasicComposition.py --manual-code 34970112332 --storage-path admin_storage.json --tests test_IDM_10_1

Bug prevalence

Whenever I do this

GitHub hash of the SDK that was being used

f09120f41f3d32678454db1915730f6f5730bb39

Platform

python, raspi

Anything else?

TC-IDM-10.1 has been failed with the message At least one cluster has failed the range and support checks for its listed attributes, commands or features

As per the log from the python script below are the issue found

[MatterTest] 11-17 11:35:18.817 INFO Problems found: [MatterTest] 11-17 11:35:18.818 INFO =============== [MatterTest] 11-17 11:35:18.818 INFO Problem: ProblemSeverity.WARNING test_name: test_IDM_10_1 location: Endpoint: 1, Cluster: 5 (0x05) Scenes, Attribute:7 (0x07) problem: Found a failure to read/decode Attribute RemainingCapacity (7, 0x0007) on Endpoint 1, Cluster Scenes (5, 0x0005) when it was claimed as supported in AttributeList ([0, 1, 2, 3, 4, 5, 6, 7, 65528, 65529, 65531, 65532, 65533]): ValueDecodeFailure(TLVValue=None, Reason=InteractionModelError(<Status.Failure: 1>)) spec_location: AttributeList Attribute [MatterTest] 11-17 11:35:18.818 INFO Problem: ProblemSeverity.WARNING test_name: test_IDM_10_1 location: Endpoint: 1, Cluster: 4294048773 (0xfff1fc05) UnitTesting, Attribute:49 (0x31) problem: Found a failure to read/decode Attribute GeneralErrorBoolean (49, 0x0031) on Endpoint 1, Cluster UnitTesting (4294048773, 0xFFF1FC05) when it was claimed as supported in AttributeList ([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 48, 49, 50, 16384, 16385, 16386, 16387, 16388, 16389, 16390, 16391, 16392, 16393, 16394, 16395, 16396, 16397, 16398, 16399, 16400, 16401, 16402, 16403, 16404, 16405, 16406, 16407, 16408, 16409, 16414, 16420, 16421, 16422, 16423, 16424, 16425, 16426, 65528, 65529, 65531, 65532, 65533]): ValueDecodeFailure(TLVValue=None, Reason=InteractionModelError(<Status.InvalidDataType: 141>)) spec_location: AttributeList Attribute [MatterTest] 11-17 11:35:18.819 INFO Problem: ProblemSeverity.WARNING test_name: test_IDM_10_1 location: Endpoint: 1, Cluster: 4294048773 (0xfff1fc05) UnitTesting, Attribute:50 (0x32) problem: Found a failure to read/decode Attribute ClusterErrorBoolean (50, 0x0032) on Endpoint 1, Cluster UnitTesting (4294048773, 0xFFF1FC05) when it was claimed as supported in AttributeList ([0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 48, 49, 50, 16384, 16385, 16386, 16387, 16388, 16389, 16390, 16391, 16392, 16393, 16394, 16395, 16396, 16397, 16398, 16399, 16400, 16401, 16402, 16403, 16404, 16405, 16406, 16407, 16408, 16409, 16414, 16420, 16421, 16422, 16423, 16424, 16425, 16426, 65528, 65529, 65531, 65532, 65533]): ValueDecodeFailure(TLVValue=None, Reason=InteractionModelError(<Status.Failure: 1>)) spec_location: AttributeList Attribute [MatterTest] 11-17 11:35:18.819 INFO Problem: ProblemSeverity.WARNING test_name: test_IDM_10_1 location: Endpoint: 2, Cluster: 5 (0x05) Scenes, Attribute:7 (0x07) problem: Found a failure to read/decode Attribute RemainingCapacity (7, 0x0007) on Endpoint 2, Cluster Scenes (5, 0x0005) when it was claimed as supported in AttributeList ([0, 1, 2, 3, 4, 5, 6, 7, 65528, 65529, 65531, 65532, 65533]): ValueDecodeFailure(TLVValue=None, Reason=InteractionModelError(<Status.Failure: 1>)) spec_location: AttributeList Attribute [MatterTest] 11-17 11:35:18.819 INFO Problem: ProblemSeverity.ERROR test_name: test_IDM_10_1 location: Endpoint: 1, Cluster: 80 (0x50) ModeSelect, Attribute:4293984256 (0xfff10000) problem: Attribute with bad prefix 65531 in cluster 80 spec_location: Manufacturer Extensible Identifier (MEI) [MatterTest] 11-17 11:35:18.819 INFO Problem: ProblemSeverity.ERROR test_name: test_IDM_10_1 location: Endpoint: 0, Cluster: 4293984256 (0xfff10000) Unknown cluster problem: Bad cluster id prefix 4293984256 spec_location: Manufacturer Extensible Identifier (MEI) [MatterTest] 11-17 11:35:18.820 INFO Problem: ProblemSeverity.ERROR test_name: test_IDM_10_1 location: Endpoint: 1, Cluster: 4293984256 (0xfff10000) Unknown cluster problem: Bad cluster id prefix 4293984256 spec_location: Manufacturer Extensible Identifier (MEI) [MatterTest] 11-17 11:35:18.820 INFO

Reference log: tc-idm-10-1.txt

cecille commented 7 months ago

Is this being run against the all clusters app?

Saravana-kr22 commented 7 months ago

Yeah it has been ran against the all-clusters-app.

cecille commented 7 months ago

This is very odd because those IDs come from zap and are therefore statically generated, but this test is run in the CI on every push. Can you give a SHA for both the test and the compiled all-clusters-app?

Saravana-kr22 commented 5 months ago

@cecille This testcases is stiil fails in the 0ff709f25f6a7af68314c1b0a147a76c30c234f6 commit

TC-IDM-10.1.txt

cecille commented 5 months ago

Ah, got it. OK, yes this is a correct failure for devices AT CERT, but it is not for example apps. That attribute is using a prefix of a test vendor ID. This is disallowed in real products, but allowed in example apps for testing. if you use a pics file with the CI value set, this condition will not trigger. But it definitely should not be allowed for devices that are going to cert. So...kind of failing correctly, you just need a pics file to tell the test to let that condition slide for example apps.