RedHatQE / teflo

Teflo is a standalone orchestration software that controls the flow of a set of testing scenarios, allowing users to provision machines, deploy software, execute tests against them and manage generated artifacts and report results.
https://teflo.readthedocs.io/en/latest/
GNU General Public License v3.0
14 stars 16 forks source link

CCITCARBON-379 Notifications with on_start triggers do not work correctly #238

Closed shay6 closed 2 years ago

shay6 commented 2 years ago

Right now, the on_start trigger works only on the first task in the scenario.

shay6 commented 2 years ago

FYI If there is no on_tasks field in notify section, then it will notify for all tasks and will be equal to:

on_tasks = ["validate", "provision", "orchestrate", "execute", "report", "cleanup"] Even if there is no such tasks in the scenario. For example, if we are not using report task in the scenario, it will try to run notify for report.

So I had to add a function to check for every task if it's exist in the scenario.

shay6 commented 2 years ago

@JunqiZhang0 @rujutashinde The below tests failed: test_cli.py::TestCli::test_remote_include test_teflo.py::TestTeflo::test_collected_group_vars

But it's not part of the code I changed! I think these unit tests are the cause of why we can't run the unit-tests more than one time. See here the issue of the unit-tests

rujutashinde commented 2 years ago

@JunqiZhang0 @rujutashinde The below tests failed: test_cli.py::TestCli::test_remote_include test_teflo.py::TestTeflo::test_collected_group_vars

But it's not part of the code I changed! I think these unit tests are the cause of why we can't run the unit-tests more than one time. See here the issue of the unit-tests

@shay6, hmm when we run unit tests here it is always cleaning up the environment/running on a fresh new environment, so it should not fail with the issue that you have pointed out. can you check to see if these unit tests pass locally? see if the unit tests are passing without your code ? the failing unit tests is just not running successfully....maybe something else is amiss

_________________________ TestCli.test_remote_include __________________________

runner = <click.testing.CliRunner object at 0x7f3940a81a58>

    @staticmethod
    def test_remote_include(runner):
        """This is for testing use of remote include"""

        """
        teflo run -s remote_include.yml --vars-data '{"hello":"unit_test"}' -t provision -t orchestrate
        """

        results = runner.invoke(
            teflo, ['run', '-s', '../assets/remote_include.yml', '--vars-data',
                    json.dumps(dict(hello='unit_test')), '-t', 'provision', '-t', 'orchestrate']
        )
>       assert results.exit_code == 0
E       assert 1 == 0
E         +1
E         -0
shay6 commented 2 years ago

@rujutashinde @JunqiZhang0 With this change, everything works as expected with this on_start feature.

I tested it with one notification and with multiple notifications. With and without on_tasks field. I executed it also with specific tasks and without, like below:

teflo run -s on_start_all.yml -t provision -t execute
teflo run -s on_start_all.yml