search

avocado-framework / avocado

Avocado is a set of tools and libraries to help with automated testing. One can call it a test framework with benefits. Native tests are written in Python and they follow the unittest pattern, but any executable can serve as a test.
https://avocado-framework.github.io/
Other
336 stars 335 forks source link

Suite / Job Configuration not respected in runnables created from recipe in runnable-recipe(s) resolver #5963

Closed clebergnu closed 1 week ago

clebergnu commented 2 weeks ago

Describe the bug Then the runnable-recipe and runnable-recipes resolvers are used with the Job API, and a configuration is supplied in the suite (or job), it's not passed on to the created runnables.

Steps to reproduce

  1. Have suite config that has a config item such as runner.identifier_format set to NOT FOO
  2. The suite config resolution should point to a file that will be resolved by the runnable-recipe resolver, such as examples/nrunner/recipes/runnable/noop.json (that lacks the runner.identifer_format config
  3. Crate a suite, with suite = TestSuite.from_config(config)
  4. With the resolver working correctly and one test being created in the suite, check for the configuration with suite.tests[0].config.get("runner.identifier_format")

The following test shows the bug in action:

class T(unittest.TestCase):
    def test_job_config_runnable_recipe_resolver_config(self):
        config = {
            "resolver.references": ["examples/nrunner/recipes/runnable/noop.json",],
            "runner.identifier_format": "NOT FOO"
        }
        suite = TestSuite.from_config(config)
        runnable = suite.tests[0]
        self.assertEqual(
            runnable.config.get("runner.identifier_format"), "NOT FOO"
        )

Expected behavior suite.tests[0].config.get("runner.identifier_format") should contain the configuration set at the suite level, that is, it should be NOT FOO.

Current behavior suite.tests[0].config.get("runner.identifier_format") == None

System information (please complete the following information):