turbot / steampipe-plugin-aws

Use SQL to instantly query AWS resources across regions and accounts. Open source CLI. No DB required.
https://hub.steampipe.io/plugins/turbot/aws
Apache License 2.0
191 stars 103 forks source link

Adding Shield Tables #2315

Closed dbermuehler closed 4 weeks ago

dbermuehler commented 2 months ago

This PR adds all relevant Shield APIs as tables to Steampipe. It will add the following new tables:

Since this is my first time contributing to this project and also the first time I worked with go, I would be more than happy to receive some feedback especially on the following points:

  1. Do you spot any obvious bugs?
  2. Is the code utilising caching, pointers, etc. efficiently?
  3. Is the way how I implemented the KeyColumns for the Get and List functions correct and can help to speed up the SQL queries?
  4. Do you agree with the division of the different APIs into the aforementioned tables?
  5. Do you agree with the names of the tables?
  6. Do you agree on the way how I split up the API JSON response in the different columns, especially in the table aws_shield_emergency_contact and aws_shield_attack_statistic
  7. Does the aws_shield_subscription table work properly if it is used with an AWS account that is not subscribed to Shield Advanced already?
  8. How can I improve the code to better follow yours or go's best practices?

Looking forward to your feedback! Cheers!

Integration test logs

Logs ``` Add passing integration test logs here ```

Example query results

Results ## `aws_shield_attack` ```sql+postgres select resource_arn, start_time, end_time from aws_shield_attack where start_time between current_date - interval '30 day' and current_date; ``` ``` +--------------------------------------------------------------+---------------------------+---------------------------+ | resource_arn | start_time | end_time | +--------------------------------------------------------------+---------------------------+---------------------------+ | arn:aws:cloudfront::XXXXXXXXXXXXX:distribution/YYYYYYYYYYYYY | 2024-10-22T09:10:00+02:00 | 2024-10-22T09:20:00+02:00 | | arn:aws:cloudfront::XXXXXXXXXXXXX:distribution/YYYYYYYYYYYYY | 2024-10-20T17:09:00+02:00 | 2024-10-20T17:18:00+02:00 | | arn:aws:cloudfront::XXXXXXXXXXXXX:distribution/YYYYYYYYYYYYY | 2024-10-06T10:33:00+02:00 | 2024-10-06T10:39:00+02:00 | +--------------------------------------------------------------+---------------------------+---------------------------+ ``` ## `aws_shield_attack_statistic` ```sql+postgres select max, unit, attack_count from aws_shield_attack_statistic order by attack_count desc; ``` ``` +--------------------+-------------------+--------------+ | max | unit | attack_count | +--------------------+-------------------+--------------+ | 6439917.8166666668 | RequestsPerSecond | 38 | +--------------------+-------------------+--------------+ ``` ## `aws_shield_drt_access` ```sql+postgres select role_arn, log_bucket_list from aws_shield_drt_access; ``` ``` +-------------------------------------------+-----------------+ | role_arn | log_bucket_list | +-------------------------------------------+-----------------+ | arn:aws:iam::XXXXXXXXXXXX:role/DRT-Role | | +-------------------------------------------+-----------------+ ``` ## `aws_shield_emergency_contact` ```sql+postgres select email_address, phone_number, contact_notes from aws_shield_emergency_contact; ``` ``` +------------------------------------+--------------+---------------------------------------------------------------------------------------------------------------------+ | email_address | phone_number | contact_notes | +------------------------------------+--------------+---------------------------------------------------------------------------------------------------------------------+ | ddos_emergency_contact@company.com | | The email address provided notifies a member of the DDoS response team of company.com who is currently on-call. | +------------------------------------+--------------+---------------------------------------------------------------------------------------------------------------------+ ``` ## `aws_shield_protection` ```sql+postgres select name, resource_arn from aws_shield_protection; ``` ``` +-------------------+--------------------------------------------------------------+ | name | resource_arn | +-------------------+--------------------------------------------------------------+ | company.com | arn:aws:route53:::hostedzone/XXXXXXXXXXXXXX | | distribution-1 | arn:aws:cloudfront::FFFFFFFFFFFF:distribution/YYYYYYYYYYYYYY | | distribution-2 | arn:aws:cloudfront::FFFFFFFFFFFF:distribution/XXXXXXXXXXXXXX | | distribution-3 | arn:aws:cloudfront::FFFFFFFFFFFF:distribution/ZZZZZZZZZZZZZZ | | distribution-4 | arn:aws:cloudfront::FFFFFFFFFFFF:distribution/AAAAAAAAAAAAAA | | distribution-5 | arn:aws:cloudfront::FFFFFFFFFFFF:distribution/BBBBBBBBBBBBBB | | distribution-6 | arn:aws:cloudfront::FFFFFFFFFFFF:distribution/CCCCCCCCCCCCCC | | distribution-7 | arn:aws:cloudfront::FFFFFFFFFFFF:distribution/EEEEEEEEEEEEEE | +-------------------+--------------------------------------------------------------+ ``` ## `aws_shield_subscription` ```sql+postgres select subscription_state, start_time, end_time, auto_renew, proactive_engagement_status from aws_shield_subscription; ``` ``` +--------------------+---------------------------+---------------------------+------------+-----------------------------+ | subscription_state | start_time | end_time | auto_renew | proactive_engagement_status | +--------------------+---------------------------+---------------------------+------------+-----------------------------+ | ACTIVE | 2023-01-01T14:59:53+02:00 | 2025-01-01T14:59:53+02:00 | ENABLED | DISABLED | +--------------------+---------------------------+---------------------------+------------+-----------------------------+ ``` ## `aws_shield_protection_group` ```sql+postgres select protection_group_id, aggregation, pattern, resource_type from aws_shield_protection_group; ``` ``` +---------------------+-------------+------------------+----------------------+ | protection_group_id | aggregation | pattern | resource_type | +---------------------+-------------+------------------+----------------------+ | Test | SUM | BY_RESOURCE_TYPE | ROUTE_53_HOSTED_ZONE | +---------------------+-------------+------------------+----------------------+ ```
rogerioacp commented 1 month ago

Hey folks, this is an awesome PR! Our team will really benefit from having this tables in Steampipe. Is there an expected deadline to merge it?

ParthaI commented 1 month ago

@misraved, could you please review this PR when you have a chance and proceed if everything looks good?

Note: Regarding the optional qualifiers (start_time and end_time) for the aws_shield_attack table, we have decided to temporarily remove support for them. Handling the various combinations of start_time and end_time with different query operators is a bit complex at the moment.

You can find the related discussion in the community channel.

Your feedback would be greatly appreciated.

Thanks!

misraved commented 1 month ago

Thanks @dbermuehler for the new set of tables đź‘Ť!!

Could you please add query results(without any sensitive information) to the PR body?

dbermuehler commented 4 weeks ago

Thank you @misraved. I added the queries, including the (anonymized) query results.