espressif / pytest-embedded

A pytest plugin that designed for embedded testing
https://docs.espressif.com/projects/pytest-embedded/en/latest/
MIT License
92 stars 26 forks source link

feat: flexible choices for run_all_single_board_cases #282

Closed horw closed 6 months ago

horw commented 6 months ago

The Unity test framework itself supports different inputs for choice tests. This merge request emulates this behavior to make it possible to control test choices with pytest-embedded.

The current ways to choose tests are:

For example:

2024-04-26 16:35:17 Here's the test menu, pick your combo:
2024-04-26 16:35:17 (1) "can use static initializers for non-POD types" [restart_init]
2024-04-26 16:35:17 (2) "static initialization guards work as expected" [misc]
2024-04-26 16:35:17 (3) "global initializers run in the correct order" [misc]
2024-04-26 16:35:17 (4) "before scheduler has started, static initializers work correctly" [misc]
2024-04-26 16:35:17 (5) "init_priority extension works" [misc]
2024-04-26 16:35:17 (6) "can use new and delete" [misc]
2024-04-26 16:35:17 (7) "can call virtual functions" [misc]
2024-04-26 16:35:17 (8) "can use std::vector" [misc]
2024-04-26 16:35:17 (9) "stack smashing protection CXX" [stack_smash]
  1. choose all (1-9): dut.run_all_single_board_cases()
  2. choose by group(2-8): dut.run_all_single_board_cases(group="misc")
  3. choose by inverting group(1,9): dut.run_all_single_board_cases(group="!misc")
  4. choose by name(9): dut.run_all_single_board_cases(names=["stack smashing protection CXX"])

addition examples: dut.run_all_single_board_cases(group="heap,mem_prot") # [heap][mem_prot] dut.run_all_single_board_cases(attributes={"test_env": "noXtal32k"}) # [test_env=noXtal32k]

horw commented 6 months ago

There are some unit test cases that use multiple groups and arguments to filter test cases, for example: [heap][mem_prot] and [test_env=noXtal32k]. So, I have added these options.