Matter (formerly Project CHIP) creates more connections between more objects, simplifying development for manufacturers and increasing compatibility for consumers, guided by the Connectivity Standards Alliance.
[CERT-TEST-FAILURE] TC-LVL-2.3 - Test Case Fails Due to First Entry in reportedCurrentLevelValuesList Not Being Larger Than startCurrentLevel at step 11 #34661
Description
The test case test_TC_LVL_2_3 fails because the first entry in reportedCurrentLevelValuesList is not larger than startCurrentLevel. According to the test plan, the first entry should be greater than startCurrentLevel when reportedCurrentLevelValuesList contains two or more entries.
Expected Behavior:
The first entry in reportedCurrentLevelValuesList should be larger than startCurrentLevel.
Actual Behavior:
The test case fails because the first entry in reportedCurrentLevelValuesList is equal to startCurrentLevel.
Reproduction steps
Command used to launch the DUT : ./chip-all-clusters-app
Command used to run the python script : python3 TC_LVL_2_3.py --commissioning-method on-network --discriminator 3840 --passcode 20202021 --storage-path admin_storage.json --paa-trust-store-path ../../credentials/development/paa-root-certs/
[MatterTest] 07-31 06:58:51.073 INFO ** Test Step 11 : If reportedCurrentLevelValuesList contains two or more entries, TH verifies the value of the first entry is larger than startCurrentLevel
[MatterTest] 07-31 06:58:51.076 ERROR Exception occurred in test_TC_LVL_2_3.
Traceback (most recent call last):
File "/home/ubuntu/Jul_Dut/connectedhomeip/no/lib/python3.10/site-packages/mobly/base_test.py", line 783, in exec_one_test
test_method()
File "/home/ubuntu/Jul_Dut/connectedhomeip/src/python_testing/matter_testing_support.py", line 1837, in per_endpoint_runner
_async_runner(body, self, args, kwargs)
File "/home/ubuntu/Jul_Dut/connectedhomeip/src/python_testing/matter_testing_support.py", line 1645, in _async_runner
return asyncio.run(runner_with_timeout)
File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run
return loop.run_until_complete(main)
File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete
return future.result()
File "/usr/lib/python3.10/asyncio/tasks.py", line 445, in wait_for
return fut.result()
File "/home/ubuntu/Jul_Dut/connectedhomeip/src/python_testing/TC_LVL_2_3.py", line 135, in test_TC_LVL_2_3
asserts.assert_greater_equal(e.value, last_value, "Values are not increasing")
File "/home/ubuntu/Jul_Dut/connectedhomeip/no/lib/python3.10/site-packages/mobly/asserts.py", line 236, in assert_greater_equal
_call_unittest_assertion(_pyunit_proxy.assertGreaterEqual,
File "/home/ubuntu/Jul_Dut/connectedhomeip/no/lib/python3.10/site-packages/mobly/asserts.py", line 52, in _call_unittest_assertion
raise signals.TestFailure(my_msg, extras=extras)
mobly.signals.TestFailure: Details=1 not greater than or equal to 5 Values are not increasing, Extras=None
[MatterTest] 07-31 06:58:51.091 INFO ***** Test Failure :
[MatterTest] 07-31 06:58:51.091 INFO Finished test in 40803913.0ms
[MatterTest] 07-31 06:58:51.092 INFO [Test] test_TC_LVL_2_3 FAIL
[MatterTest] 07-31 06:58:51.104 INFO Summary for test class TC_LVL_2_3: Error 0, Executed 1, Failed 1, Passed 0, Requested 1, Skipped 0
[MatterTest] 07-31 06:58:51.106 INFO Summary for test run MatterTest@07-31-2024_06-58-08-503:
Total time elapsed 42.594418749038596s
Artifacts are saved in "/tmp/matter_testing/logs/MatterTest/07-31-2024_06-58-08-503"
Test summary saved in "/tmp/matter_testing/logs/MatterTest/07-31-2024_06-58-08-503/test_summary.yaml"
Test results: Error 0, Executed 2, Failed 1, Passed 1, Requested 2, Skipped 0
Feature Area
Other
Test Case
TC-LVL-2.3
Description The test case test_TC_LVL_2_3 fails because the first entry in reportedCurrentLevelValuesList is not larger than startCurrentLevel. According to the test plan, the first entry should be greater than startCurrentLevel when reportedCurrentLevelValuesList contains two or more entries.
PFA test plan screenshot :
https://github.com/CHIP-Specifications/chip-test-plans/blob/master/src/cluster/levelcontrol.adoc#tc-lvl-2-3-reporting-requirements-with-dut_server
Expected Behavior: The first entry in reportedCurrentLevelValuesList should be larger than startCurrentLevel.
Actual Behavior: The test case fails because the first entry in reportedCurrentLevelValuesList is equal to startCurrentLevel.
Reproduction steps
Command used to launch the DUT :
./chip-all-clusters-app
Command used to run the python script :python3 TC_LVL_2_3.py --commissioning-method on-network --discriminator 3840 --passcode 20202021 --storage-path admin_storage.json --paa-trust-store-path ../../credentials/development/paa-root-certs/
Bug prevalence
Everytime
GitHub hash of the SDK that was being used
86011093e730609a7dca462f8cc0dcf278019c56
Platform
raspi
PFA log Below : TC-LVL-2.3.txt