search

project-chip / certification-tool

A test harness and tooling designed to simplify development, testing, and certification for devices, guided by the Connectivity Standards Alliance.
https://csa-iot.org/
Apache License 2.0
28 stars 15 forks source link

Docker has to be updated by adding the data model directory #198

Open Survensa opened 5 months ago

Survensa commented 5 months ago

Describe the bug

To run the TC_AccessChecker.py test, need to include the XML files from the data model directory. But it's not available in current docker.

So the data model from ~/certification-tool/backend/test_collections/matter/sdk_tests/sdk_checkout/python_testing$ has to be added inside the docker root@ubuntu:~/python_testing#.

Steps to reproduce the behavior

  1. Build the docker: docker run -v /var/credentials:/credentials -v /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket -v /launch_dir --privileged --network host -it connectedhomeip/chip-cert-bins:9f6d627e0262e1d023986291948bb4e845be803e
  2. Change the directory to python_testing

Environment

connectedhomeip/chip-cert-bins:9f6d627e0262e1d023986291948bb4e845be803e

ccruzagralopes commented 5 months ago

Hi @Survensa, you can do that by adding the volume mapping to your command: -v /home/ubuntu/certification-tool/backend/test_collections/matter/sdk_tests/sdk_checkout/python_testing/data_model:/root/python_testing/data_model

$ docker run -v /var/credentials:/credentials -v /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket -v /home/ubuntu/certification-tool/backend/test_collections/matter/sdk_tests/sdk_checkout/python_testing/data_model:/root/python_testing/data_model -v /launch_dir --privileged --network host -it connectedhomeip/chip-cert-bins:9f6d627e0262e1d023986291948bb4e845be803e
root@vm-matter-x64:~# cd python_testing/
root@vm-matter-x64:~/python_testing# ls
MinimalRepresentation.py      TC_DeviceConformance.py      TC_MWOM_1_2.py         TC_RVCRUNM_2_2.py       TC_VALCC_4_5.py
TC_ACE_1_2.py                 TC_EEM_2_1.py                TC_OPCREDS_3_1.py      TC_SC_3_6.py            TC_pics_checker.py
TC_ACE_1_3.py                 TC_EEM_2_2.py                TC_OPSTATE_1_1.py      TC_TIMESYNC_2_1.py      TestBatchInvoke.py
TC_ACE_1_4.py                 TC_EEM_2_3.py                TC_OPSTATE_2_1.py      TC_TIMESYNC_2_10.py     TestCommissioningTimeSync.py
TC_ACE_1_5.py                 TC_EEM_2_4.py                TC_OPSTATE_2_2.py      TC_TIMESYNC_2_11.py     TestConformanceSupport.py
TC_ACL_2_2.py                 TC_EEM_2_5.py                TC_OPSTATE_2_3.py      TC_TIMESYNC_2_12.py     TestGroupTableReports.py
TC_AccessChecker.py           TC_EEVSE_2_2.py              TC_OPSTATE_2_4.py      TC_TIMESYNC_2_13.py     TestMatterTestingSupport.py
TC_BOOLCFG_2_1.py             TC_EEVSE_2_4.py              TC_OPSTATE_2_5.py      TC_TIMESYNC_2_2.py      TestSpecParsingSupport.py
TC_BOOLCFG_3_1.py             TC_EEVSE_2_5.py              TC_OVENOPSTATE_1_1.py  TC_TIMESYNC_2_4.py      TestTimeSyncTrustedTimeSource.py
TC_BOOLCFG_4_1.py             TC_EEVSE_Utils.py            TC_OVENOPSTATE_2_1.py  TC_TIMESYNC_2_5.py      basic_composition_support.py
TC_BOOLCFG_4_2.py             TC_EPM_2_1.py                TC_OVENOPSTATE_2_2.py  TC_TIMESYNC_2_6.py      conformance_support.py
TC_BOOLCFG_4_3.py             TC_EPM_2_2.py                TC_OVENOPSTATE_2_3.py  TC_TIMESYNC_2_7.py      data_model
TC_BOOLCFG_4_4.py             TC_EnergyReporting_Utils.py  TC_OVENOPSTATE_2_4.py  TC_TIMESYNC_2_8.py      drlk_2_x_common.py
TC_BOOLCFG_5_1.py             TC_FAN_3_1.py                TC_OVENOPSTATE_2_5.py  TC_TIMESYNC_2_9.py      global_attribute_ids.py
TC_BOOLCFG_5_2.py             TC_FAN_3_2.py                TC_OpstateCommon.py    TC_TIMESYNC_3_1.py      hello_external_runner.py
TC_CGEN_2_4.py                TC_FAN_3_3.py                TC_PWRTL_2_1.py        TC_TestEventTrigger.py  hello_test.py
TC_CNET_4_4.py                TC_FAN_3_4.py                TC_RR_1_1.py           TC_VALCC_2_1.py         matter_testing_support.py
TC_DA_1_2.py                  TC_FAN_3_5.py                TC_RVCCLEANM_1_2.py    TC_VALCC_3_1.py         mdns_discovery
TC_DA_1_5.py                  TC_ICDM_2_1.py               TC_RVCCLEANM_2_1.py    TC_VALCC_3_2.py         requirements.txt
TC_DA_1_7.py                  TC_IDM_1_2.py                TC_RVCCLEANM_2_2.py    TC_VALCC_3_3.py         spec_parsing_support.py
TC_DGGEN_2_4.py               TC_IDM_1_4.py                TC_RVCOPSTATE_2_1.py   TC_VALCC_3_4.py         taglist_and_topology_test_support.py
TC_DRLK_2_12.py               TC_IDM_4_2.py                TC_RVCOPSTATE_2_3.py   TC_VALCC_4_1.py         test_testing
TC_DRLK_2_2.py                TC_MWOCTRL_2_2.py            TC_RVCOPSTATE_2_4.py   TC_VALCC_4_2.py
TC_DRLK_2_3.py                TC_MWOCTRL_2_3.py            TC_RVCRUNM_1_2.py      TC_VALCC_4_3.py
TC_DeviceBasicComposition.py  TC_MWOCTRL_2_4.py            TC_RVCRUNM_2_1.py      TC_VALCC_4_4.py
root@vm-matter-x64:~/python_testing# cd data_model/
root@vm-matter-x64:~/python_testing/data_model# ls
README.md  clusters  device_types  scraper_version  spec_sha
ccruzagralopes commented 5 months ago

Actually, you'll need to add the following mapping: -v /home/ubuntu/certification-tool/backend/test_collections/matter/sdk_tests/sdk_checkout/python_testing:/root/python_testing. Then, to run the tests, you'll have to navigate to python_testing/scripts/sdk to find the scripts.

ccruzagralopes commented 5 months ago

This mapping solution should be ok for this release. But, for the next one, we should update the chip-cert-bins Dockerfile to include the data model files in the SDK image, instead of copying the files to TH backend and then mapping the volume during the test execution, like it's done today.

ccruzagralopes commented 5 months ago

Opened the PR on the SDK repo: https://github.com/project-chip/connectedhomeip/pull/32361

ccruzagralopes commented 5 days ago

Related SDK PR: https://github.com/project-chip/connectedhomeip/pull/34481