ccruzagralopes
commented
8 months ago @Saravana-kr22 can you please share the steps you took and the commands used to run the test directly in the docker container?
Closed Saravana-kr22 closed 5 months ago
ccruzagralopes
commented
8 months ago @Saravana-kr22 can you please share the steps you took and the commands used to run the test directly in the docker container?
Saravana-kr22
commented
8 months ago @ccruzagralopes here is the steps and command used to run in the docker container:
Step 1: Run the docker container docker run -v $PATH_TO_credentials:/credentials -v /var/run/dbus/system_bus_socket:/var/run/dbus/system_bus_socket -v $(pwd):/launch_dir --privileged --network host -it connectedhomeip/chip-cert-bins:548e4f3de32053abe44854c35882bbc354f0538a
Step 2: Advertise the dut
Step 3: Run the testcase in the docker using the command python3 python_testing/TC_RR_1_1.py --commissioning-method on-network --discriminator 3840 --passcode 20202021 --storage-path admin_storage.json --paa-trust-store-path /credentials/development/paa-root-certs/
ccruzagralopes
commented
8 months ago I get the same error when trying to run the test using TH UI, but I'm facing an error when trying to run it directly: TC_RR_1_1-error.log
ccruzagralopes
commented
8 months ago @cecille can you please take a look at this issue? Do you know of anything that could cause this particular test case to fail with TypeError: 'ValueDecodeFailure' object is not iterable when running in TH but pass when running directly in the command line?
cecille
commented
8 months ago It's not a value decode failure that's causing the failure, it's a timeout sending a command
atterTest] 01-11 18:01:41.770 INFO Time out! failed to receive invoke command response from Exchange: 4001i
[MatterTest] 01-11 18:01:41.771 ERROR Exception occurred in test_TC_RR_1_1.
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/mobly/base_test.py", line 783, in exec_one_test
test_method()
File "/root/python_testing/matter_testing_support.py", line 1339, in async_runner
return asyncio.run(runner_with_timeout)
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
return fut.result()
File "/root/python_testing/TC_RR_1_1.py", line 479, in test_TC_RR_1_1
Dict[int, Clusters.GroupKeyManagement.Structs.GroupInfoMapStruct]] = await self.add_all_groups(
File "/root/python_testing/TC_RR_1_1.py", line 657, in add_all_groups
add_response: Clusters.Groups.Commands.AddGroupResponse = await client.SendCommand(
File "/usr/local/lib/python3.10/dist-packages/chip/ChipDeviceCtrl.py", line 902, in SendCommand
return await future
chip.exceptions.ChipStackError: src/app/CommandSender.cpp:298: CHIP Error 0x00000032: Timeout
[MatterTest] 01-11 18:01:41.774 INFO ***** Test Failure :
[MatterTest] 01-11 18:01:41.774 INFO Finished test in 18540ms
[MatterTest] 01-11 18:01:41.774 INFO [Test] test_TC_RR_1_1 ERRORoh, sorry, the timeout was in your log. The top error is different.
cecille
commented
8 months ago Do you have DUT logs? It basically means the TH failed to read the ACL attribute from the device.
Saravana-kr22
commented
6 months ago This testacase is failed with the same error in the latest TH : Version: 2.10-beta3+spring2024 Sha: 1fd1429
TH log: rr-1-1.log DUT log: tc-rr-1-1dut.txt
Saravana-kr22
commented
6 months ago This testcase is passing in the connectedhomeip/chip-cert-bins docker
TH log: rr-1-1(docker).txt DUT log: rr-1-1(dut).txt
ccruzagralopes
commented
6 months ago I've run the test case in TH and manually using --trace-to json:log to get more logs:
During the failing step (Step 3b: Validating ACL entry for fabric 1):
In the manual logs, we can see that we get the expected data:
In the TH logs, we get kUnsupportedAccess:
ccruzagralopes
commented
6 months ago @cecille do you have any ideia why there's a difference in these executions?
cecille
commented
6 months ago It's this:
if __name__ == "__main__":
default_matter_test_main(maximize_cert_chains=True, controller_cat_tags=[0x0001_0001])When the test is run on its own, the default controller is created with cat tags, and that's what's used to retain access to the acl cluster.
When it's run though the TH, it's not set up to do that. And because of the way those got crammed in there, there's no good way to pass them through. We'll have to make some changes to allow those to be set up by the TH. But actually, if we want this to be able to be run as a set, we should just cram those onto the cert for every commissioning.
ccruzagralopes
commented
6 months ago But actually, if we want this to be able to be run as a set, we should just cram those onto the cert for every commissioning.
I think that would be the best option, because we'd like to commission once and then run multiple test cases.
cecille
commented
6 months ago @ccruzagralopes - Carolina, would you mind trying out a PR in the TH to make sure it fixes the issue for the TH? https://github.com/project-chip/connectedhomeip/pull/32468
ccruzagralopes
commented
6 months ago Not at all, I'll test it. Thanks!
ccruzagralopes
commented
6 months ago I tried it out and that ValueDecodeFailure doesn't happen anymore, but I'm back to the timeout I was facing before.
UI_Test_Run_2024_03_08_16_35_00.log
Maybe that's some problem with my environment. I'm using a VM right now, I'll try it out with a Raspberry Pi.
ccruzagralopes
commented
6 months ago Got the same result running in a Raspberry Pi
Describe the bug
TC-RR-1.1 is failed with the
TypeError: 'ValueDecodeFailure' object is not iterable. The docker used for Execution of the SDK python testcases needs to be updated to hadle these type of issue.Log files
tc-rr-1-1.log
Additional Information
Note : This testcases is failed only in the UI but passed in the connectedhomeip/chip-cert-bins docker.