search

project-chip / certification-tool

A test harness and tooling designed to simplify development, testing, and certification for devices, guided by the Connectivity Standards Alliance.
https://csa-iot.org/
Apache License 2.0
36 stars 22 forks source link

[Bug] Command line arguments needs to be updated for the SDK Python testcases in UI #125

Closed Saravana-kr22 closed 7 months ago

Saravana-kr22 commented 8 months ago

Describe the bug

Command line arguments are missing in the UI SDK Python testcases. This cause the failure the testcasses

For ex: The TC-ACE-1.4 testcase is failed with the error PIXIT.ACE.APPENDPOINT must be included on the command line in the --int-arg flag as PIXIT.ACE.APPENDPOINT:<endpoint> this due to the missing argument --int-arg PIXIT.ACE.APPENDPOINT::1

log: tc-ace-1-4.log

For the below mentioned test cases are failed due to the missing command line arguments in UI. Hence the python command needs to be updated.

rquidute commented 8 months ago

Hi @Saravana-kr22 could you please add the missing arguments in Project Config,following the example bellow:

 "test_parameters": {
      "paa-trust-store-path": "/paa-root-certs",
      "storage-path": "/root/admin_storage.json",
      "int-arg": "PIXIT.ACE.APPENDPOINT:1 PIXIT.ACE.APPDEVTYPEID:256",
      "string-arg": "PIXIT.ACE.APPCLUSTER:OnOff PIXIT.ACE.APPATTRIBUTE:OnOff"
    }
Screenshot 2023-12-14 at 12 35 31 Screenshot 2023-12-14 at 12 32 44
Saravana-kr22 commented 8 months ago

Hi @rquidute I have tried with the above test-parameters but the arguments werenot passed to the script with the message like Argument paa-trust-store-path is not valid and the test cases is failed due to this.

image

This is the project config I used for execution:

{ "config": { "network": { "wifi": { "ssid": "testharness", "password": "wifi-password" }, "thread": { "rcp_serial_path": "/dev/ttyACM0", "rcp_baudrate": 115200, "on_mesh_prefix": "fd11:22::/64", "network_interface": "eth0", "dataset": { "channel": "15", "panid": "0x1234", "extpanid": "1111111122222222", "networkkey": "00112233445566778899aabbccddeeff", "networkname": "DEMO" }, "otbr_docker_image": null } }, "dut_config": { "discriminator": "3840", "setup_code": "20202021", "pairing_mode": "onnetwork", "chip_tool_timeout": null, "chip_tool_use_paa_certs": false }, "test_parameters": { "paa-trust-store-path": "/paa-root-certs/", "storage-path": "/root/admin_storage.json", "int-arg": "PIXIT.ACE.APPENDPOINT:1 PIXIT.ACE.APPDEVTYPEID:256", "string-arg": "PIXIT.ACE.APPCLUSTER:OnOff PIXIT.ACE.APPATTRIBUTE:OnOff" } }, "pics": { "clusters": {} } }

rquidute commented 8 months ago

Oh I see the problem @Saravana-kr22 . This will be fixed in next release. But you can still run in the current version if you change the test arguments by replacing the - to _ for example: from paa-trust-store-path to paa_trust_store_path, storage-path to storage_path and so on. In the next release you should use argument in the way it is expected by the python test, for example: paa-trust-store-path and storage-path.

hiltonlima commented 8 months ago

Fixed test_parameters section

 "test_parameters": {
      "paa_trust_store_path": "/paa-root-certs/",
      "storage_path": "/root/admin_storage.json",
      "int_arg": "PIXIT.ACE.APPENDPOINT:1 PIXIT.ACE.APPDEVTYPEID:256",
      "string_arg": "PIXIT.ACE.APPCLUSTER:OnOff PIXIT.ACE.APPATTRIBUTE:OnOff"
    }
raju-apple commented 7 months ago

@Saravana-kr22 were you able to verify if the config section CLI arguments work fine for you now ?

Saravana-kr22 commented 7 months ago

This issue has been resolved in latest image Version: 2.10-beta2+spring2024