[TC-IDM-10.2] ICD Management requirements wrongly defined for Matter 1.4

doublemis1 commented 3 months ago

Summary Title:

ICD Management requirements wrongly defined for Matter 1.4. The new features are still in provisional state.

Description:

Test failed if the device support LIT and other ICDM features which for Matter 1.3 were provisional.

Logs:


[MatterTest] 08-14 08:12:42.870 INFO 
Problem: ProblemSeverity.ERROR
    test_name: IDM-10.2
    location: 
       Endpoint: 0,
       Cluster:  70 (0x46) IcdManagement
      Attribute:65532 (0xfffc)
    problem: Disallowed feature with mask 0x01
    spec_location: 

[MatterTest] 08-14 08:12:42.871 INFO 
Problem: ProblemSeverity.ERROR
    test_name: IDM-10.2
    location: 
       Endpoint: 0,
       Cluster:  70 (0x46) IcdManagement
      Attribute:65532 (0xfffc)
    problem: Disallowed feature with mask 0x02
    spec_location: 

[MatterTest] 08-14 08:12:42.871 INFO 
Problem: ProblemSeverity.ERROR
    test_name: IDM-10.2
    location: 
       Endpoint: 0,
       Cluster:  70 (0x46) IcdManagement
      Attribute:65532 (0xfffc)
    problem: Disallowed feature with mask 0x04
    spec_location: 

[MatterTest] 08-14 08:12:42.872 INFO 
Problem: ProblemSeverity.ERROR
    test_name: IDM-10.2
    location: 
       Endpoint: 0,
       Cluster:  70 (0x46) IcdManagement
      Attribute:6 (0x06)
    problem: Attribute 0x06 is included, but is disallowed by conformance. Conformance: P, UAT, implemented features: CIP,UAT,LITS
    spec_location: 

[MatterTest] 08-14 08:12:42.872 INFO 
Problem: ProblemSeverity.ERROR
    test_name: IDM-10.2
    location: 
       Endpoint: 0,
       Cluster:  70 (0x46) IcdManagement
      Attribute:8 (0x08)
    problem: Attribute 0x08 is included, but is disallowed by conformance. Conformance: P, LITS, implemented features: CIP,UAT,LITS
    spec_location:

naveenkommareddi commented 3 months ago

@cecille

naveenkommareddi commented 2 months ago

Per Yinyi Hu "This error is expected because ICD is still provisional till SVE. During TE2, we should ran with "bool-arg": "allow_provisional:true" We should do the same for SVE. This will not be an issue when the ICD feature eventually becomes non-provisional

doublemis1 commented 2 months ago

does the information about additional param is documented somewhere? It will be the best to provide the list of expected test parameters which need to be set

cecille commented 2 months ago

No - we don't generally have a good solution for exposing test parameters. This one is extra complicated because it's definitely disallowed during certification so it needs to be checked by the ATLs.

We have a list of common test failures here: https://github.com/project-chip/matter-test-scripts/blob/main/docs/common_test_failures/TC-IDM-10.2.md. If you want to add that info to the documentation, it would be welcome. The TH team has an ongoing issue to expose that documentation in the TH.

cjandhyala commented 2 months ago

@doublemis1 usually VS doc list out all python script commands along with the parameters that needs to be passed. The only Catch is , we should not allow to use this parameter during certification runs, so we will have to update the VS after SVE.

project-chip / matter-test-scripts