anderseknert
commented
2 years ago Have not found any official way of obtaining this data. Very few seem to have done this before, and those I found have scraped the AWS documentation pages.
We could do this too of course, and I did a quick POC getting this data using both Python and Rego. See this gist for details. It works by making a first call to the table of contents for the cloudformation resources page, then picks all the categories from there and makes a call to each corresponding .json file for the resources under that category name. This is just how their website works, although normally each category is queried only when you click to expand it. When run as a script, ~190 calls need to be made to AWS to extract all data. I have no idea whether this is considered tolerable or not.
Another option would be to download the CFN JSON schemas for any region which is likely to have the most supported resource types, and then extract the typeName from each file. This would require a single request only, but some more processing on our side. I think I'll try this.
While we can add these manually to styra-opa-hook.json during development as we add more and more resource type example policies, we'll want to list all resource types available for AWS Cloudformation before we publish this hook to the marketplace. The reason for this is that we'll want to keep the hook as generic as possible, and have the user determine whether something is allowed or not in policy rather than configuration.
Ideally, I think we'll have a script run nightly (or whatever interval seems appropriate) to fetch the resource definitions from the AWS API and make sure we're up to date. If we aren't we should create a PR and later on a release from that (this part could be done manually).