Feat/add expected tag values

rinzool commented 1 year ago

How

Create a new file controls/expected_tag_values.sp similar to mandatory.sp
For each tag in variable expected_tag_values, we check, for all resources possessing the tag, if the value is one of the listed value

Result

$ steampipe check benchmark.expected_tag_values --var='expected_tag_values={"Environment": ["Production", "Staging"]}'

Expected Tag Values ...................................................................................................................................................... 5 / 6 [==========]
| 
+ Access Analyzer analyzers should have appropriate tag values ........................................................................................................... 0 / 0 [          ]
| 
+ API Gateway stages should have appropriate tag values .................................................................................................................. 0 / 0 [          ]
| 
+ CloudFront distributions should have appropriate tag values ............................................................................................................ 0 / 0 [          ]
| 
+ CloudTrail trails should have appropriate tag values ................................................................................................................... 0 / 0 [          ]
| 
+ CloudWatch alarms should have appropriate tag values ................................................................................................................... 0 / 0 [          ]
| 
+ CloudWatch log groups should have appropriate tag values ............................................................................................................... 1 / 2 [====      ]
| | 
| ALARM: /aws/test-log-group2 has a wrong value for tag Environment. "prod" must be one of ["Production", "Staging"]. ................................................ 
| OK   : /aws/test-log-group1 has a good value for tag Environment. ..................................................................................................

aminvielledebatAtBedrock commented 1 year ago

Ping @misraved :) Do you plan to merge it ?

dim-ops commented 1 year ago

Great feature, I need of it ! @misraved

e-gineer commented 1 year ago

Validation of tag values is definitely a valuable addition to this mod - thanks @rinzool! In fact, I think it's so valuable that it would be good to discuss the cases a little further so we get the feature right. I've expanded the discussion on the issue #26 with some of those ideas / use cases / etc - seemed more appropriate than dumping it here in the PR(?).

rinzool commented 1 year ago

Hi @e-gineer, I just pushed a commit to add the use of LIKE operator It works as expected, but it may increase a lot the duration of queries if there are a lot of wildcards used (without any wildcards it's very fast :+1:) I used the json_array_elements_text function to explode expected values so they can be compared to real tag value using LIKE, and then I group by resources and aggregated the tag comparison using bool_or operator. I did not find any other (and faster) solution, but if anyone has a better idea, please tell me! :pray: :grinning:

rinzool commented 1 year ago

Hi @e-gineer @misraved, did you find some time to look at this PR / Issue? :pray:

rinzool commented 1 year ago

Thanks @cbruno10! You were write we needed a double backslash to escape wildcard \\_ And thanks for the suggestion of better descriptions and naming :+1:

cbruno10 commented 1 year ago

Hey @rinzool , I was using the controls earlier today and had a few additional questions/suggestions:

Rows for resources

When expected_tag_values = {} (which is also the default), if I run a control, I get 0 rows back. If I compare that to the other controls, I get back a row per resource in the results:

Is it possible to update the query so the controls always return a row for each resource? If expected_tag_values is set to {}, then I think the control should return OK, as there are no unexpected tag values for the specified tag keys (which there are none according to the variable).

Multiple unexpected tag values

If I have multiple tag key/values in expected_tag_values and a resource has both of those tag keys with incorrect values, e.g.,

expected_tag_values = {
  "Environment": ["Prod", "Staging", "Dev"]
  "Name": ["test"]
}

cbruno@M1P tagsawsvalues % steampipe query "select tags from aws_ec2_instance"
+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
| tags                                                                                                                                                      |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------+
| {"Environment":"dev","Name":"App Dev","TestApps":"MyApplication","cost_center":"111111","department":"IT","key":"myprivatekey"}                           |
+-----------------------------------------------------------------------------------------------------------------------------------------------------------+

When I run the control, only one result shows one key:

Maybe if there's more than 1 tag with unexpected values, we could shorten the reason so it's something like i-1234 has unexpected tag values for tag keys: ["Environment", "Name"]. We could try and show the expected values for each, but I suspect that will get too long.

Thoughts?

rinzool commented 1 year ago

Hi @cbruno10, I like your ideas! I just committed them, can you please see it it works fine for you? :pray:

I added the display of possible values, but you are right it makes a very long line.. :confused: Can you try it and tell me what you think please?

cbruno10 commented 1 year ago

@rinzool Thanks for making the changes, I ran the controls again for various scenarios and agree that sometimes the reasons are long, especially when the keys/values are long.

Based on your changes, here are my suggestions for new reasons:

OK state:
- When all tags for a resource have expected tag values or do not have the expected tag keys: i-123 has expected tag values.
- When resource has no tags: i-123 has no tags.
- When no expected tag values are set: i-123 has tags but no expected tag values are set.
Alarm state:
- When one or more tag for a resource has an unexpected tag value: i-123 has unexpected tag values for tags: Name, Environment.

In general, these are simpler than the current reasons, but I found the current ones to be a bit long (even the one in OK state), e.g.,

+ EC2 instances should have appropriate tag values ................................................................................................ 3 / 4 [==========]
  |
  ALARM: App Dev has unexpected tag values for tag keys: Environment,Name. Expected values: ["Prod", "Staging", "Dev"],["Demo"]. .............. us-east-1 123456789012
  ALARM: App Staging has unexpected tag values for tag keys: Environment,Name. Expected values: ["Prod", "Staging", "Dev"],["Demo"]. .......... us-east-1 123456789012
  ALARM: Demo has unexpected tag values for tag keys: Environment. Expected values: ["Prod", "Staging", "Dev"]. ............................... us-east-1 123456789012
  OK   : i-0bd0fba70e1c158c9 has expected tag values or no tag values for tag keys: Environment,Name,StrictCostCenter,created_by. ............. us-east-1 123456789012

If a user needs more info, in particular in the alarm case, then they can manually compare the resources' tags and the expected tag values.

Thoughts?

rinzool commented 1 year ago

And voilà @cbruno10 :tada: I updated the messages with your ideas, I think it's quite explicit but not too long now :ok_hand:

cbruno10 commented 1 year ago

@rinzool The new reasons look great, thanks again for your work on this PR!

turbot / steampipe-mod-aws-tags