RedHatQE / teflo

Teflo is a standalone orchestration software that controls the flow of a set of testing scenarios, allowing users to provision machines, deploy software, execute tests against them and manage generated artifacts and report results.
https://teflo.readthedocs.io/en/latest/
GNU General Public License v3.0
14 stars 16 forks source link

CCITCARBON-356 More strict validation of schemas in teflo #227

Closed shay6 closed 2 years ago

shay6 commented 2 years ago

If the ssh_key file is not exist like below: ssh_key: "keys/notRealFile"

The stdout will look like this:

2022-05-23 17:10:57,883 INFO ** BLASTER BEGIN **
2022-05-23 17:10:57,911 INFO Validating <class 'teflo.resources.scenario.Scenario'> (beaker resource)
2022-05-23 17:10:57,913 INFO Validating <class 'teflo.resources.assets.Asset'> (beaker-node)
2022-05-23 17:10:57,914 INFO Beaker config already exists, skip creation.
2022-05-23 17:10:57,959 ERROR Failed to validate beaker-node
2022-05-23 17:10:57,961 ERROR A exception was raised while processing task: beaker-node method: run
Traceback (most recent call last):
  File "/home/shshevac/.virtualenvs/teflo/lib/python3.8/site-packages/blaster/blast.py", line 83, in run
    value = getattr(task_obj, method)()
  File "/home/shshevac/work/teflo/teflo/tasks/validate.py", line 54, in run
    self.resource.validate()
  File "/home/shshevac/work/teflo/teflo/resources/assets.py", line 540, in validate
    getattr(AssetProvisioner(self), 'validate')()
  File "/home/shshevac/work/teflo/teflo/provisioners/asset_provisioner.py", line 71, in validate
    self.plugin.validate()
  File "/home/shshevac/work/teflo/teflo/provisioners/ext/bkr_client_plugin/beaker_client_plugin.py", line 354, in validate
    schema_validator(schema_data=self.build_profile(self.asset), schema_files=[self.__schema_file_path__],
  File "/home/shshevac/work/teflo/teflo/helpers.py", line 384, in schema_validator
    c.validate(raise_exception=True)
  File "/home/shshevac/.virtualenvs/teflo/lib/python3.8/site-packages/pykwalify/core.py", line 183, in validate
    self._start_validate(self.source)
  File "/home/shshevac/.virtualenvs/teflo/lib/python3.8/site-packages/pykwalify/core.py", line 230, in _start_validate
    self._validate(value, root_rule, path, done)
  File "/home/shshevac/.virtualenvs/teflo/lib/python3.8/site-packages/pykwalify/core.py", line 264, in _validate
    self._validate_mapping(value, rule, path, done=None)
  File "/home/shshevac/.virtualenvs/teflo/lib/python3.8/site-packages/pykwalify/core.py", line 604, in _validate_mapping
    self._validate(v, r, u"{0}/{1}".format(path, k), done)
  File "/home/shshevac/.virtualenvs/teflo/lib/python3.8/site-packages/pykwalify/core.py", line 266, in _validate
    self._validate_scalar(value, rule, path, done=None)
  File "/home/shshevac/.virtualenvs/teflo/lib/python3.8/site-packages/pykwalify/core.py", line 659, in _validate_scalar
    self._handle_func(value, rule, path, done)
  File "/home/shshevac/.virtualenvs/teflo/lib/python3.8/site-packages/pykwalify/core.py", line 287, in _handle_func
    ret = method(value, rule, path)
  File "/home/shshevac/work/teflo/teflo/provisioners/ext/bkr_client_plugin/extensions.py", line 36, in valid_file_exist
    raise AssertionError(
AssertionError: ssh_key must be an existing file! Cannot find file: keys/notRealFile.
2022-05-23 17:10:57,963 INFO ** BLASTER COMPLETE **
2022-05-23 17:10:57,963 INFO     -> TOTAL DURATION: 0h:0m:0s
shay6 commented 2 years ago

I added also a validation for the kickstart field, because it's the same check as ssh-key Now, it's ready for review @rujutashinde @JunqiZhang0

shay6 commented 2 years ago

I changed only that word in the schema s/whitboard/whiteboard , this is the only addition I made to the last push

shay6 commented 2 years ago

I set a wrong ssh-key path: I removed the provision stage and the validate check is working as expected I removed the validate stage and run the job only with provision stage, and also the provision sends error as expected