Make twister dig deeper in directory structure to find additional .yaml files

[jhn-nordic]

Is your enhancement proposal related to a problem? Please describe. We want to add application/sample specific tests in projects. When adding a my_sample /tests/ folder side by side with the my_sample/src/ folder of a project we experience that twister cannot find the our testcase.yaml files inside the /tests/ folder as it stops searching further down the folder structure when it finds a .yaml file. In this case /my_sample/sample.yaml The reason for having the application specific tests together with the application and not together with the other tests is that we want to be able to make out of tree copies of the application where the user can update the tests together with changes they make in the application.

Describe the solution you'd like I would like twister to be modified somehow to dig deeper to find my testcase.yaml I would like to keep directory structure as proposed.

Describe alternatives you've considered Either dig into all folders, or possibly make a hint in the my_sample/sample.yaml that it should look for more files. It should be scalable in such a way that you do not have to change any central configuration to make twister find the test file such as e.g add -T samples/my_sample/tests to the twister call

Additional context

[jhn-nordic]

@PerMac @nashif @carlescufi

[nashif]

I am not clear exactly what the ask is and why is this useful.

[jhn-nordic]

The ask is to make it possible for twister to find .yaml files in subfolders in a samples. The use case is to have sample specific tests in a subfolder within a sample. We want CI to find these tests automatically with twister. In its current form twister will stop searching at the root level of the sample when it finds the sample.yml

The reason for having tests bundled with the sample in the sample folder is that we want customers to take the entire folder out of tree and customize both the sample and the tests for their own needs. Here is an example where we add tests: https://github.com/nrfconnect/sdk-nrf/pull/4173 We want our CI to pick up these tests without specifying the exact path. That will get ugly when we add more samples and applications with their own tests.

Here we have an out of tree setup of the asset tracker v2 application: https://github.com/NordicSemiconductor/asset-tracker-cloud-firmware

[jhn-nordic]

@nashif did I manage to explain it?

[jorgenmk]

Created a PR for this issue: #34041

[carlescufi]

PR merged, closing issue.