search

project-chip / connectedhomeip

Matter (formerly Project CHIP) creates more connections between more objects, simplifying development for manufacturers and increasing compatibility for consumers, guided by the Connectivity Standards Alliance.
https://buildwithmatter.com
Apache License 2.0
7.45k stars 1.99k forks source link

[CERT-TEST-FAILURE] TC-LVL-2.3 - Test Case Fails Due to First Entry in reportedCurrentLevelValuesList Not Being Larger Than startCurrentLevel at step 11 #34661

Closed Rajashreekalmane closed 2 months ago

Rajashreekalmane commented 2 months ago

Feature Area

Other

Test Case

TC-LVL-2.3

Description The test case test_TC_LVL_2_3 fails because the first entry in reportedCurrentLevelValuesList is not larger than startCurrentLevel. According to the test plan, the first entry should be greater than startCurrentLevel when reportedCurrentLevelValuesList contains two or more entries.

PFA test plan screenshot : image

https://github.com/CHIP-Specifications/chip-test-plans/blob/master/src/cluster/levelcontrol.adoc#tc-lvl-2-3-reporting-requirements-with-dut_server

Expected Behavior: The first entry in reportedCurrentLevelValuesList should be larger than startCurrentLevel.

Actual Behavior: The test case fails because the first entry in reportedCurrentLevelValuesList is equal to startCurrentLevel.

Reproduction steps

Command used to launch the DUT : ./chip-all-clusters-app Command used to run the python script : python3 TC_LVL_2_3.py --commissioning-method on-network --discriminator 3840 --passcode 20202021 --storage-path admin_storage.json --paa-trust-store-path ../../credentials/development/paa-root-certs/

[MatterTest] 07-31 06:58:51.073 INFO ** Test Step 11 : If reportedCurrentLevelValuesList contains two or more entries, TH verifies the value of the first entry is larger than startCurrentLevel [MatterTest] 07-31 06:58:51.076 ERROR Exception occurred in test_TC_LVL_2_3. Traceback (most recent call last): File "/home/ubuntu/Jul_Dut/connectedhomeip/no/lib/python3.10/site-packages/mobly/base_test.py", line 783, in exec_one_test test_method() File "/home/ubuntu/Jul_Dut/connectedhomeip/src/python_testing/matter_testing_support.py", line 1837, in per_endpoint_runner _async_runner(body, self, args, kwargs) File "/home/ubuntu/Jul_Dut/connectedhomeip/src/python_testing/matter_testing_support.py", line 1645, in _async_runner return asyncio.run(runner_with_timeout) File "/usr/lib/python3.10/asyncio/runners.py", line 44, in run return loop.run_until_complete(main) File "/usr/lib/python3.10/asyncio/base_events.py", line 649, in run_until_complete return future.result() File "/usr/lib/python3.10/asyncio/tasks.py", line 445, in wait_for return fut.result() File "/home/ubuntu/Jul_Dut/connectedhomeip/src/python_testing/TC_LVL_2_3.py", line 135, in test_TC_LVL_2_3 asserts.assert_greater_equal(e.value, last_value, "Values are not increasing") File "/home/ubuntu/Jul_Dut/connectedhomeip/no/lib/python3.10/site-packages/mobly/asserts.py", line 236, in assert_greater_equal _call_unittest_assertion(_pyunit_proxy.assertGreaterEqual, File "/home/ubuntu/Jul_Dut/connectedhomeip/no/lib/python3.10/site-packages/mobly/asserts.py", line 52, in _call_unittest_assertion raise signals.TestFailure(my_msg, extras=extras) mobly.signals.TestFailure: Details=1 not greater than or equal to 5 Values are not increasing, Extras=None [MatterTest] 07-31 06:58:51.091 INFO ***** Test Failure : [MatterTest] 07-31 06:58:51.091 INFO Finished test in 40803913.0ms [MatterTest] 07-31 06:58:51.092 INFO [Test] test_TC_LVL_2_3 FAIL [MatterTest] 07-31 06:58:51.104 INFO Summary for test class TC_LVL_2_3: Error 0, Executed 1, Failed 1, Passed 0, Requested 1, Skipped 0 [MatterTest] 07-31 06:58:51.106 INFO Summary for test run MatterTest@07-31-2024_06-58-08-503: Total time elapsed 42.594418749038596s Artifacts are saved in "/tmp/matter_testing/logs/MatterTest/07-31-2024_06-58-08-503" Test summary saved in "/tmp/matter_testing/logs/MatterTest/07-31-2024_06-58-08-503/test_summary.yaml" Test results: Error 0, Executed 2, Failed 1, Passed 1, Requested 2, Skipped 0

Bug prevalence

Everytime

GitHub hash of the SDK that was being used

86011093e730609a7dca462f8cc0dcf278019c56

Platform

raspi

PFA log Below : TC-LVL-2.3.txt

Rajashreekalmane commented 2 months ago

Test case is passing in latest Master TOT, hence closing this issue TC-LVL-2.3.txt