General Compliance Features

[dhutchison]

(Going to be updating this issue as I come up with a bit of a design)

Part of the original contents of the PR #257 included functionality to define regex patterns for naming conventions. This was born out of an approach that I am using at work for this type of compliance check in the templates we produce. Although what we do has the downside that we have a common test file copied (and getting out of sync) between repositories.

@shadycuz suggested in this comment that a more flexible approach could be taken to include more compliance functionality by providing a hook system that could be imported like a plugin. This would allow for checking other resource type specific things consistently like ensuring S3 buckets always had encryption etc.

This branches into the kind of functionality that I have used CloudFormation Guard for in projects, but with the ability to use the python language instead of the custom DSL for guard (and that cloud-radar has better parameter support than cfn-guard at this point in time).

Requirements wise we need:

• the ability to set multiple hooks per resource type
• the ability to set hooks that apply to all resources

These types of hooks should be able to be ran against a template pre-render:

• properties
• mappings

These should be able to be ran post-render:

• resources
• outputs

Use cases:

• thing for checking resources start with r, outputs o etc (there is a link somewhere to a best practice guide that mentions this)
• all templates must contain these x parameters (e.g. lifecycle, environment)
• testing names meet a pattern
• testing that resources that support tags have given tags

For checking a resource, need:

• resource name
• resource data

Need some way via metadata to mark a rule as excluded

[dhutchison]

315 contains an initial implementation supporting locally defined hooks.

For the plugin side of things, the Python Packaging User Guide lays out 3 approaches to supporting plugins. Options 1 (Using naming convention) and 3 (Using package metadata) seem like the simpler ones to support & use.

[shadycuz]

@dhutchison I was going to show you the flake8 way of doing it https://github.com/PyCQA/flake8/blob/main/src/flake8/plugins/finder.py#L176 but I agree the Python Packaging guide seems way more straightforward. I don't think we need to support both 1 and 3. We could just support a method. Which ever one you prefer =). Thanks <3

[dhutchison]

I think flake8 seems similar to option 3 - I've got a branch taking that approach already but I've never tested it.

I'm starting on a reference plugin implementation based on some of the rules from cfn-nag so they are realistic examples. Hopefully get that implemented over the next few weeks (free time depending), then plugin support will be tested