search

zephyrproject-rtos / zephyr

Primary Git Repository for the Zephyr Project. Zephyr is a new generation, scalable, optimized, secure RTOS for multiple hardware architectures.
https://docs.zephyrproject.org
Apache License 2.0
10.1k stars 6.2k forks source link

Ztest: Unexpected behavior in the ztest shell. #75322

Closed gbarkadiusz closed 1 day ago

gbarkadiusz commented 1 week ago

Describe the bug While using the ztest shell with the ztest run-testcase <testsuite::testcase> command sometimes it runs all testsuite instead of the particular testcase.

To Reproduce Steps to reproduce the behavior:

  1. cd zephyrproject/zephyr
  2. source zephyr-env.sh
  3. west build -b qemu_x86 -p -t run tests/kernel/semaphore/semaphore/ -- -DCONFIG_ZTEST_SHELL=y
  4. run ztest run-testcase semaphore::test_k_sem_init multiple times.

Expected behavior Running the particular testcase.

Impact Annoyance

Logs and console output


uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.009 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.002 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.002 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.002 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.002 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
uart:~$ ztest run-testcase semaphore::test_k_sem_init
Running TESTSUITE semaphore
===================================================================
START - test_k_sem_init
 PASS - test_k_sem_init in 0.001 seconds
===================================================================
START - test_sem_count_get
 PASS - test_sem_count_get in 0.002 seconds
===================================================================
START - test_sem_give_from_isr
 PASS - test_sem_give_from_isr in 0.001 seconds
===================================================================
START - test_sem_give_from_thread
 PASS - test_sem_give_from_thread in 0.001 seconds
===================================================================
START - test_sem_give_take_from_isr
 PASS - test_sem_give_take_from_isr in 0.001 seconds
===================================================================
START - test_sem_measure_timeout_from_thread
 PASS - test_sem_measure_timeout_from_thread in 0.001 seconds
===================================================================
START - test_sem_measure_timeouts
 PASS - test_sem_measure_timeouts in 1.005 seconds
===================================================================
START - test_sem_multi_take_timeout_diff_sem
 PASS - test_sem_multi_take_timeout_diff_sem in 5.007 seconds
===================================================================
START - test_sem_multiple_threads_wait
 PASS - test_sem_multiple_threads_wait in 2.040 seconds
===================================================================
START - test_sem_reset
 PASS - test_sem_reset in 0.109 seconds
===================================================================
START - test_sem_reset_waiting
 PASS - test_sem_reset_waiting in 0.018 seconds
===================================================================
START - test_sem_take_multiple
 PASS - test_sem_take_multiple in 1.012 seconds
===================================================================
START - test_sem_take_no_wait
 PASS - test_sem_take_no_wait in 0.002 seconds
===================================================================
START - test_sem_take_no_wait_fails
 PASS - test_sem_take_no_wait_fails in 0.001 seconds
===================================================================
START - test_sem_take_timeout
 PASS - test_sem_take_timeout in 0.001 seconds
===================================================================
START - test_sem_take_timeout_fails
 PASS - test_sem_take_timeout_fails in 0.544 seconds
===================================================================
START - test_sem_take_timeout_forever
 PASS - test_sem_take_timeout_forever in 0.110 seconds
===================================================================
START - test_sem_thread2isr
 PASS - test_sem_thread2isr in 0.001 seconds
===================================================================
START - test_sem_thread2thread
 PASS - test_sem_thread2thread in 0.001 seconds
===================================================================
TESTSUITE semaphore succeeded
Running TESTSUITE semaphore_1cpu
===================================================================
START - test_sem_multiple_take_and_timeouts
 PASS - test_sem_multiple_take_and_timeouts in 1.255 seconds
===================================================================
START - test_sem_queue_mutual_exclusion
 PASS - test_sem_queue_mutual_exclusion in 0.117 seconds
===================================================================
START - test_sem_take_timeout_isr
 PASS - test_sem_take_timeout_isr in 0.001 seconds
===================================================================
TESTSUITE semaphore_1cpu succeeded
Running TESTSUITE semaphore_null_case
===================================================================
START - test_sem_count_get_null
E: 0 is not a valid k_sem
E: address is not a known kernel object
E: syscall z_vrfy_k_sem_count_get failed check: access denied
E: EAX: 0x00000000, EBX: 0x00000000, ECX: 0x00000000, EDX: 0x00000000
E: ESI: 0x00000000, EDI: 0x00000000, EBP: 0x00000000, ESP: 0x007e9fb4
E: EFLAGS: 0x00000246 CS: 0x002b CR3: 0x0013e300
E: call trace:
E: EIP: 0x00100578
E: NULL base ptr
E: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
E: Current thread: 0x135000 (unknown)
Caught system error -- reason 3 1
Fatal error expected as part of test case.
 PASS - test_sem_count_get_null in 0.003 seconds
===================================================================
START - test_sem_give_null
E: 0 is not a valid k_sem
E: address is not a known kernel object
E: syscall z_vrfy_k_sem_give failed check: access denied
E: EAX: 0x00000000, EBX: 0x00000000, ECX: 0x00000000, EDX: 0x00000000
E: ESI: 0x00000000, EDI: 0x00000000, EBP: 0x00000000, ESP: 0x007e9fb4
E: EFLAGS: 0x00000246 CS: 0x002b CR3: 0x0013e300
E: call trace:
E: EIP: 0x0010060d
E: NULL base ptr
E: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
E: Current thread: 0x135000 (unknown)
Caught system error -- reason 3 1
Fatal error expected as part of test case.
 PASS - test_sem_give_null in 0.002 seconds
===================================================================
START - test_sem_init_null
E: 0 is not a valid k_sem
E: address is not a known kernel object
E: syscall z_vrfy_k_sem_init failed check: access denied
E: EAX: 0x00000000, EBX: 0x00000000, ECX: 0x00000000, EDX: 0x00000000
E: ESI: 0x00000000, EDI: 0x00000000, EBP: 0x00000000, ESP: 0x007e9fb4
E: EFLAGS: 0x00000246 CS: 0x002b CR3: 0x0013e300
E: call trace:
E: EIP: 0x0010067d
E: NULL base ptr
E: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
E: Current thread: 0x135000 (unknown)
Caught system error -- reason 3 1
Fatal error expected as part of test case.
 PASS - test_sem_init_null in 0.002 seconds
===================================================================
START - test_sem_reset_null
E: 0 is not a valid k_sem
E: address is not a known kernel object
E: syscall z_vrfy_k_sem_reset failed check: access denied
E: EAX: 0x00000000, EBX: 0x00000000, ECX: 0x00000000, EDX: 0x00000000
E: ESI: 0x00000000, EDI: 0x00000000, EBP: 0x00000000, ESP: 0x007e9fb4
E: EFLAGS: 0x00000246 CS: 0x002b CR3: 0x0013e300
E: call trace:
E: EIP: 0x0010065d
E: NULL base ptr
E: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
E: Current thread: 0x135000 (unknown)
Caught system error -- reason 3 1
Fatal error expected as part of test case.
 PASS - test_sem_reset_null in 0.002 seconds
===================================================================
START - test_sem_take_null
E: 0 is not a valid k_sem
E: address is not a known kernel object
E: syscall z_vrfy_k_sem_take failed check: access denied
E: EAX: 0x00000000, EBX: 0x00000000, ECX: 0x00000000, EDX: 0x00000000
E: ESI: 0x00000000, EDI: 0x00000000, EBP: 0x00000000, ESP: 0x007e9fb0
E: EFLAGS: 0x00000246 CS: 0x002b CR3: 0x0013e300
E: call trace:
E: EIP: 0x0010059a
E: NULL base ptr
E: >>> ZEPHYR FATAL ERROR 3: Kernel oops on CPU 0
E: Current thread: 0x135000 (unknown)
Caught system error -- reason 3 1
Fatal error expected as part of test case.
 PASS - test_sem_take_null in 0.002 seconds
===================================================================
TESTSUITE semaphore_null_case succeeded

Environment (please complete the following information):

golowanow commented 1 week ago

While using the ztest shell with the ztest run-testcase <testsuite::testcase> command sometimes it runs all testsuite instead of the particular testcase.

It seems because of memory leak at Ztest shell: after several repeats it can't properly allocate a string for name, so NULL value is treated to run all test cases.

@gbarkadiusz, please try with #75370

gbarkadiusz commented 1 week ago

https://github.com/zephyrproject-rtos/zephyr/pull/75370 Fixes it.