Describe the bug
I have multiple regions defined in .taskcat.yaml, let's say us-east-1 on project level and us-east-2, us-west-1 and us-west-2 specifics tests. taskcat lint seem to use all regions defined in any level but if an error or warning occurs, the region name is not part of the output. In my case, I have a list of allowed EC2 Instances, all of them are supported in us-east-1, us-east-2 and us-west-2, but some are not available in us-west-1. In that case, taskcat lint throws a lot of warnings but I can't see in the output, that it's about the region us-west-1, which makes troubleshooting a little complicated.
To Reproduce
Steps to reproduce the behavior:
Are you testing a QuickStart or Custom template? - Partner Solution (in development)
Attach or link a copy of the template if possible (remove any sensitive info) - Not public yet
Provide the parameters that you passed. (remove any sensitive info) - see end of the issue
How did you install taskcat? (docker or pip3) - pip
Are you using a profile, an instance role or access keys to run taskcat? - profile
Is your AWS environment configured via aws configure? - no, isengard
Expected behavior
I would expect that taskcat provides the name of the region a specific error or warning belongs to. Even better would be, if the region can be set by a cli parameter or only regions on project level / the default test is taken into account if not configured otherwise.
Screenshots
n/a
**Version (Please make sure you are running the latest version of taskcat)
Taskcat Version: 0.9.41
Python Version (python3 required): 3.9.16
Additional context
I can technically configure to ignore instance type related warnings, but in case of a typo or something, it feels valuable to run an actual test. But 100% coverage of all instance types across all regions is not realistic, so testing just against one specific region, as sanity check that in general the list is correct, feels like a good approach.
Passed parameters
This is an example of a instance type allowed values list that works for example with us-east-1, because c6id is available in that region, and will throw warnings with us-west-1, which is expected and fine. But that I don't seem to have visibility in the taskcat lint output and can't limit the regions that should be included is something that could be improved.
[WARN ][WARN ] : line 62 [2030] [Check if parameters have a valid value] You must specify a valid allowed value for InstanceType
(c6id.large). Valid values are ["c1.medium", "c1.xlarge",
"c3.2xlarge", "c3.4xlarge", "c3.8xlarge", "c3.large", "c3.xlarge",
"c4.2xlarge", "c4.4xlarge", "c4.8xlarge", "c4.large", "c4.xlarge",
"c5.12xlarge", "c5.18xlarge", "c5.24xlarge", "c5.2xlarge",
"c5.4xlarge", "c5.9xlarge", "c5.large", "c5.metal", "c5.xlarge", ........... ]
Describe the bug I have multiple regions defined in
.taskcat.yaml
, let's sayus-east-1
onproject
level andus-east-2
,us-west-1
andus-west-2
specifics tests.taskcat lint
seem to use all regions defined in any level but if an error or warning occurs, the region name is not part of the output. In my case, I have a list of allowed EC2 Instances, all of them are supported inus-east-1
,us-east-2
andus-west-2
, but some are not available inus-west-1
. In that case,taskcat lint
throws a lot of warnings but I can't see in the output, that it's about the regionus-west-1
, which makes troubleshooting a little complicated.To Reproduce Steps to reproduce the behavior:
aws configure
? - no, isengardExpected behavior I would expect that
taskcat
provides the name of the region a specific error or warning belongs to. Even better would be, if the region can be set by a cli parameter or only regions on project level / the default test is taken into account if not configured otherwise.Screenshots n/a
**Version (Please make sure you are running the latest version of taskcat)
Additional context I can technically configure to ignore instance type related warnings, but in case of a typo or something, it feels valuable to run an actual test. But 100% coverage of all instance types across all regions is not realistic, so testing just against one specific region, as sanity check that in general the list is correct, feels like a good approach.
Passed parameters
This is an example of a instance type allowed values list that works for example with
us-east-1
, becausec6id
is available in that region, and will throw warnings withus-west-1
, which is expected and fine. But that I don't seem to have visibility in thetaskcat lint
output and can't limit the regions that should be included is something that could be improved.Warning