aws-ia / taskcat

Test all the CloudFormation things! (with TaskCat)
https://aws-ia.github.io/taskcat/
Apache License 2.0
1.17k stars 213 forks source link

Mismatched region OR complete lack of named test OR complete lack of .taskcat.yml gives no output at all #743

Closed DarwinJS closed 1 year ago

DarwinJS commented 2 years ago

Describe the bug When a test profile and a region is specified on the command line and

  1. the test profile does not have the region in it's list, taskcat just exits with no output.
  2. the same is true if the test is not found
  3. the same is true if the entire file .taskcat.yml is not found

Since it is doing a validation which failed, it would be helpful to emit it as an error.

To Reproduce Steps to reproduce the behavior:

  1. Do a test run on a test and specify --region with a region that is not specified in the test definition in .taskcat.yml OR specify a test name that does not exist in .taskcat.yml
  2. Observe that it simply exits

Expected behavior An error that says that the CLI specified region does not match any of the regions specified in the test.

Screenshots If applicable, add screenshots to help explain your problem.

**Version (Please make sure you are running the latest version of taskcat)

To find versions: Via taskcat: taskcat -V Via pip3: pip3 show taskcat

Note: both version should match

To update taskcat run: for docker : docker pull taskcat/taskcat for pip3: pip3 install --upgrade taskcat

Additional context Add any other context about the problem here.

tlindsay42 commented 2 years ago

Per our separate discussion, the error messages and documentation around these scenarios should be improved.