canonical / hotsos

Software analysis toolkit. Define checks in high-level language and leverage library to perform analysis of common Cloud applications.
Apache License 2.0
31 stars 37 forks source link

The YAML test files do not raise error when the corresponding scenario file is absent #819

Closed mustafakemalgilor closed 4 months ago

mustafakemalgilor commented 4 months ago

When the test with the name ".../tests/scenarios/openstack/keystone/foo.yaml" exists, but the scenario ".../scenarios/openstack/keystone/foo.yaml" is absent, the test proceeds to evaluate the conditions anyway. This is the same with the explicitly specified target-name property as well:

lp1896125-17-0-1.yaml

target-name: lp1896125-no-such-file.yaml
data-root:
  files:
    sos_commands/dpkg/dpkg_-l: |
      ii  keystone 2:17.0.1-0ubuntu0.20.04.1 amd64
    var/log/keystone/keystone.log: |
      Traceback (most recent call last):
          File "/usr/lib/python3.10/contextlib.py", line 135, in __enter__
            return next(self.gen)
          File "/usr/lib/python3/dist-packages/ldappool/__init__.py", line 391, in connection
            raise MaxConnectionReachedError(self.uri)
      ldappool.MaxConnectionReachedError: ldaps://ldapserver.example.com
raised-bugs:
  # none

Test results:

scenario/lp1896125-rework 16:23 > tox -e py3 -- lp1896125-17-0-1

{0} tests.unit.test_openstack.TestOpenstackScenarios.test_keystone_bugs_lp1896125-17-0-1 [0.005862s] ... ok

======
Totals
======
Ran: 1 tests in 0.0059 sec.
 - Passed: 1
 - Skipped: 0
 - Expected Fail: 0
 - Unexpected Success: 0
 - Failed: 0
Sum of execute time for each test: 0.0059 sec.

==============
Worker Balance
==============
 - Worker 0 (1 tests) => 0:00:00.005862
  py3: OK (0.85=setup[0.02]+cmd[0.83] seconds)
  congratulations :) (0.88 seconds)