search

avocado-framework / avocado

Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test.
https://avocado-framework.github.io/
Other
336 stars 335 forks source link

Extra flexibility for the default test suite class for custom schedulers #5954

Closed pevogam closed 3 weeks ago

pevogam commented 3 weeks ago

Discussed in https://github.com/avocado-framework/avocado/discussions/5949

Originally posted by **pevogam** June 10, 2024 Hi all, we have a small plugin that tries to integrate better with avocado and reuse avocado-run and avocado-list as much as possible with our custom scheduling mechanism. Our scheduler is refused with the error `Suite creation for runner "traverser" is not supported`. Note that if I do ```diff diff --git a/avocado/core/suite.py b/avocado/core/suite.py index 823ae05eb..c91587a43 100644 --- a/avocado/core/suite.py +++ b/avocado/core/suite.py @@ -340,7 +340,7 @@ class TestSuite: config.update(job_config) config.update(suite_config) runner = config.get("run.suite_runner") - if runner == "nrunner": + if runner in ["nrunner", "traverser"]: suite = cls._from_config_with_resolver(config, name) if suite.test_parameters or suite.variants: suite.tests = suite._get_test_variants() ``` then everything works smoothly with no problem of any kind which made me wonder why this value is hardcoded there. Could we make this configurable perhaps? I am aware we can inherit via a separate test suite class but it seemed like an overkill for what is needed, especially considering the above would work just fine. Is there perhaps a better way to provide a custom scheduler with the default avocado classes? At present I have to rely on the following to make the above work: ```diff diff --git a/avocado_i2n/plugins/auto.py b/avocado_i2n/plugins/auto.py index 580e6c92..6af1d454 100644 --- a/avocado_i2n/plugins/auto.py +++ b/avocado_i2n/plugins/auto.py @@ -76,5 +76,36 @@ class Auto(CLI): config["params"] = [] cmd_parser.params_from_cmd(config) + # TODO: crude override of the avocado test suite due to + # hardcoded suite runner + from avocado.core.suite import TestSuite, TestSuiteError + def from_config(config, name=None, job_config=None): + suite_config = config + config = settings.as_dict() + if job_config: + config.update(job_config) + config.update(suite_config) + runner = config.get("run.suite_runner") + # TODO: hardcoded runner type, this method works perfectly fine for + # our own traverser scheduler (suite runner) + if runner in ["nrunner", "traverser"]: + suite = TestSuite._from_config_with_resolver(config, name) + if suite.test_parameters or suite.variants: + suite.tests = suite._get_test_variants() + else: + raise TestSuiteError( + f'Suite creation for runner "{runner}" ' f"is not supported" + ) + + if not config.get("run.ignore_missing_references"): + if not suite.tests: + msg = ( + "Test Suite could not be created. No test references " + "provided nor any other arguments resolved into tests" + ) + raise TestSuiteError(msg) + + return suite + TestSuite.from_config = from_config log.debug(f"Setting test runner to 'traverser'") config["run.suite_runner"] = "traverser" ``` and is all needed in order to overwrite the hard-coded value. Disclaimer: We have our scheduler work just fine from our own CLICmd plugin but the above is an approach that should enable better integration with avocado's own run/list plugins.