[Bug] Missing Module 'chip.testing' During Test Execution for Fabric Synchronization Cluster and Ecosystem Information Cluster Python Test Cases

[Rajashreekalmane]

Describe the bug

During the execution of test cases from the Fabric Synchronization Cluster and Ecosystem Information Cluster, the tests failed due to a missing Python module. Specifically, the error No module named 'chip.testing' was encountered.

WARNING | 2024-10-24 10:30:44.246247 | Test Failure: Python test step failure: No module named 'chip.testing'

Ecosystem Information Cluster:

• [ ] TC-ECOINFO-2.1
• [ ] TC-ECOINFO-2.2

Fabric Synchronization Cluster:

• [ ] TC-MCORE.FS-1.1
• [ ] TC-MCORE.FS-1.2
• [ ] TC-MCORE.FS-1.3
• [ ] TC-MCORE.FS-1.4
• [ ] TC-MCORE.FS-1.5

Steps to reproduce the behavior

Execute any of the affected test cases using the following command (example for TC_MCORE_FS_11): Terminal 1 : sudo rm -rf /tmp/chip* && ./fabric-admin Terminal 2 : ./fabric-bridge-app

Once the fabric bridge app connected with fabric admin,

2. From the Test-Harness, select the test case.

   • Choose the TC-MCORE.FS-1.1 Python test case for execution.
     • Specify arguments for execution via the UI (use product-specific values):
   3. In the UI, under "test_parameters," input the following: "string-arg": "th_server_app_path:/home/ubuntu/apps/chip-all-clusters-app", "endpoint": "0"

3. Start the test case: Click the Start button on the Test-Harness user interface to begin the execution.

Expected behavior

No response

Log files

MCOREFS and ECOSYS.zip

PICS file

No response

Screenshots

Environment

TH Version: v2.11+fall2024 TH SHA: 8f86ccd TH SDK SHA: f2e5de7

Additional Information

No response

[fabiowmm]

Scripts TC-MCORE.FS-1.1 and TC-MCORE.FS-1.4 need to be changed to fix the importing of tasks.py.

[andy31415]

I believe test harness needs to install the chip-testing WHL (this is probably missing).

[fabiowmm]

@andy31415 all the needed files are in TH, the problem is the path. I believe you guys use a virtual env in the CI, but TH never used it, thus the path differences.

[arkq]

I will look into it, but the path should not be the case here as long as the chip-testing is installed properly (it does not have to be installed in virt env).

[fabiowmm]

@arkq TH just uses the chip-cert-bins docker image as is. And has always been like that.

[arkq]

TH just uses the chip-cert-bins docker image as is. And has always been like that.

Yes, I can see that, I will have to add patch for the chip-cert-bins image, which will properly install chip-testing package, if that's acceptable. It's not possible to fix this on the script side in a "clean" way. Right now I'm trying to build the TH image on my side to see how it looks like after the build.

[fabiowmm]

my concern is such a change would call for thorough re-testing of TH and today is the Matter v1.4 package release deadline. Can we go with the dirty workaround for now and fix this for coming releases?

CC @cecille

[arkq]

my concern is such a change would call for thorough re-testing of TH and today is the Matter v1.4 package release deadline.

OK. So, how about "patching" current chip-cert-bins? Just use current chip-cert-bins image as a base, e.g.:

FROM chip-cert-bins:latest
RUN cp -R /root/python_testing/scripts/sdk/matter_testing_infrastructure/chip <path where chip python modules are istalled>

I can verify that and get proper destination after I will build chip-cert-bins on my side (or maybe I can download it from somewhere?) Alternative solution might be to run such copy (in a docker) before running one of the affected test scripts.

[fabiowmm]

let me add you to the conversation in slack.