Reported by Inaky Perez-Gonzalez:
Test cases shall be classified according to a taxonomy that offers multiple categories (represented by tags) to which the test cases have to belong.
This task describes what categories are available and how test cases need to fit into said category classification.
Test cases should be classified according to a TBD taxonomy; this task describes what categories are available and how test cases need to fit into said category classification.
Use cases
The intention is to create categories that can be used for (non conclusive list):
CI to decide what is run on each commit, release, nightly runs; eg:
build and run all testcases with tag bat_submit
developers to know what has to be run locally before submitting; for example, a developer working on the core GPIO subsystem to add a feature mostly found on the STM32x family might run:
build and run all testcases tagged subsys_gpio soc_stm32x
and then later on, might want to run on all platforms to verify nothing was broken:
build and run all testcases tagged subsys_gpio
Taxonomy
The classification shall be done by tagging testcases with sets of {{TAG}}, with both values clarified by the specification laid out herein.
Initial tree (to be discussed):
tags bat_XYZ: Basic Acceptance Tests; values:
-- submit: Test cases to be ran per each submission to CI
-- daily: Test cases to be ran once a day on the master branch
-- release: Test cases to be ran on reach pre-release and release
tag subsys_XYZ: affected subsystem
VALUE: {core/irq,core/...,network/ip,network/BLE,...}; maybe append /DRIVER for driver specific tests?
tag arch/soc/platform: which architectures (x86, arc, arm, mips...), SOCs (Quark*, STM32x, ...) or platforms it applies to.
This also gives us tools to create support to guess which tags should be run based on files modified.
nashif
commented
8 years ago
Reported by Inaky Perez-Gonzalez: Test cases shall be classified according to a taxonomy that offers multiple categories (represented by tags) to which the test cases have to belong.
This task describes what categories are available and how test cases need to fit into said category classification.
Test cases should be classified according to a TBD taxonomy; this task describes what categories are available and how test cases need to fit into said category classification.
Use cases
The intention is to create categories that can be used for (non conclusive list):
CI to decide what is run on each commit, release, nightly runs; eg:
developers to know what has to be run locally before submitting; for example, a developer working on the core GPIO subsystem to add a feature mostly found on the STM32x family might run:
and then later on, might want to run on all platforms to verify nothing was broken:
Taxonomy
The classification shall be done by tagging testcases with sets of {{TAG}}, with both values clarified by the specification laid out herein.
Initial tree (to be discussed):
This also gives us tools to create support to guess which tags should be run based on files modified.
(Imported from Jira INF-72)