Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test.
Is your feature request related to a problem? Please describe.
The exeter framework integrates with Avocado by producing output that the "runnables-recipe" resolver can understand (JSON representations of Runnables).
But, that requires an intermediate step, that is, the executable test files have to be executed in a special way (currently with --avocado) so that JSON is produced, and can then be consumed by Avocado's "runnables-recipe" resolver.
Describe the solution you'd like
It'd be convenient to not require the intermediate step (although it should still be an option). The most straightforward solution is to have a new resolver configuration for the format of the command line that will produce the manifest of tests.
Describe alternatives you've considered
The solution described above has the drawback of requiring the execution of binaries/scripts that may not be completely trusted.
The resolver plugins that ship with Avocado do not execute any untrusted code (for instance an avocado list $(path-to-test-repo) won't execute any of the tests found. Avocado's safeloader does a static analysis for Python code, which could be extended. For shell based tests, it could also read some hints from the code. For binaries, maybe reading data added to an ELF section?
Is your feature request related to a problem? Please describe. The exeter framework integrates with Avocado by producing output that the "runnables-recipe" resolver can understand (JSON representations of Runnables).
But, that requires an intermediate step, that is, the executable test files have to be executed in a special way (currently with
--avocado) so that JSON is produced, and can then be consumed by Avocado's "runnables-recipe" resolver.Describe the solution you'd like It'd be convenient to not require the intermediate step (although it should still be an option). The most straightforward solution is to have a new resolver configuration for the format of the command line that will produce the manifest of tests.
A configuration such as:
Describe alternatives you've considered The solution described above has the drawback of requiring the execution of binaries/scripts that may not be completely trusted.
The resolver plugins that ship with Avocado do not execute any untrusted code (for instance an
avocado list $(path-to-test-repo)won't execute any of the tests found. Avocado's safeloader does a static analysis for Python code, which could be extended. For shell based tests, it could also read some hints from the code. For binaries, maybe reading data added to an ELF section?