turbot / steampipe-plugin-aws

Use SQL to instantly query AWS resources across regions and accounts. Open source CLI. No DB required.
https://hub.steampipe.io/plugins/turbot/aws
Apache License 2.0
191 stars 103 forks source link

Handle the "not found" error code in the getBucketTagging and getBucketWebsite functions Closes #2334 #2335

Closed ParthaI closed 1 week ago

ParthaI commented 3 weeks ago

Integration test logs

Logs ``` Add passing integration test logs here ```

Example query results

Results Breaks: ``` > select * from aws_s3_bucket Error: aws: operation error S3: GetBucketWebsite, https response error StatusCode: 404, RequestID: 5F42KQGXEHF4XCKR, HostID: RjigpSaozYqUmuu33Jlgl1GdLxm811ewXfw7TvnyMBqdDCplwqJ+Peu8sr4j8iHGCoSiCEZ5gGo=, api error NoSuchWebsiteConfiguration: The specified bucket does not have a website configuration (SQLSTATE HV000) +------+-----+---------------+-------------------------+--------------------+-----------------------+-------------------+---------------------+--------------------+-------------------------+-----------------------> | name | arn | creation_date | bucket_policy_is_public | versioning_enabled | versioning_mfa_delete | block_public_acls | block_public_policy | ignore_public_acls | restrict_public_buckets | event_notification_con> +------+-----+---------------+-------------------------+--------------------+-----------------------+-------------------+---------------------+--------------------+-------------------------+-----------------------> +------+-----+---------------+-------------------------+--------------------+-----------------------+-------------------+---------------------+--------------------+-------------------------+-----------------------> ``` ``` Error: aws: operation error S3: GetBucketTagging, https response error StatusCode: 404, RequestID: 3RJ6DYJGY806SNDJ, HostID: 5As1rq9QSVXOG9ng+dteO9h0x168M4kG1rpW8qVMhCJr6nm4xInUFNC0bwQZSNCdjXS+zR+ks7I=, api error NoSuchTagSet: The TagSet does not exist (SQLSTATE HV000) +------+-----+---------------+-------------------------+--------------------+-----------------------+-------------------+---------------------+--------------------+-------------------------+-----------------------> | name | arn | creation_date | bucket_policy_is_public | versioning_enabled | versioning_mfa_delete | block_public_acls | block_public_policy | ignore_public_acls | restrict_public_buckets | event_notification_con> +------+-----+---------------+-------------------------+--------------------+-----------------------+-------------------+---------------------+--------------------+-------------------------+-----------------------> +------+-----+---------------+-------------------------+--------------------+-----------------------+-------------------+---------------------+--------------------+-------------------------+-----------------------> ``` Fixed: ``` > select * from aws_s3_bucket +-----------------------------------------------------------------+------------------------------------------------------------------------------+---------------------------+-------------------------+-------------> | name | arn | creation_date | bucket_policy_is_public | versioning_e> +-----------------------------------------------------------------+------------------------------------------------------------------------------+---------------------------+-------------------------+-------------> | flowpipe-test-8a6db914-9a98-f789-e9d2-d13df22528ee | arn:aws:s3:::flowpipe-test-8a6db914-9a98-f789-e9d2-d13df22528ee | 2024-10-09T13:02:36+07:00 | false | false > | aws-security-data-lake-us-east-1-qeydzsin6buhruyudnvt7gnkpmxck3 | arn:aws:s3:::aws-security-data-lake-us-east-1-qeydzsin6buhruyudnvt7gnkpmxck3 | 2022-12-01T17:06:58+07:00 | false | true > | aws-cloudtrail-logs-xxxxxxxxxxxx-5edd7283 | arn:aws:s3:::aws-cloudtrail-logs-xxxxxxxxxxxx-5edd7283 | 2023-06-22T19:39:07+07:00 | false | false > | aws-security-data-lake-ap-southeast-2-4i3pnujz7bpnx72x6oheacrcx | arn:aws:s3:::aws-security-data-lake-ap-southeast-2-4i3pnujz7bpnx72x6oheacrcx | 2022-12-01T17:07:29+07:00 | false | false > | test-s3-bucket-2600a5ba-da1e-d998-32e4-d20fa84662b1 | arn:aws:s3:::test-s3-bucket-2600a5ba-da1e-d998-32e4-d20fa84662b1 | 2024-10-01T23:24:56+07:00 | false | false > | elasticbeanstalk-us-east-1-xxxxxxxxxxxx | arn:aws:s3:::elasticbeanstalk-us-east-1-xxxxxxxxxxxx | 2021-08-24T17:09:29+07:00 | false | false > | test-cloudtrail-s3-bucket-f0117bbb-370d-f1cf-9997-5f56625d347b | arn:aws:s3:::test-cloudtrail-s3-bucket-f0117bbb-370d-f1cf-9997-5f56625d347b | 2024-10-01T23:48:39+07:00 | false | false > | fp-2024-05-23-10-19-02 | arn:aws:s3:::fp-2024-05-23-10-19-02 | 2024-05-23T17:19:06+07:00 | false | false > | flowpipe-test-64c48ea5-e9f6-77e6-ae11-0694e9c962bf | arn:aws:s3:::flowpipe-test-64c48ea5-e9f6-77e6-ae11-0694e9c962bf | 2024-10-09T13:00:54+07:00 | false | false > | test-cloudtrail-s3-bucket-3d364669-8ca1-50d5-9e1b-db8ba3fd4289 | arn:aws:s3:::test-cloudtrail-s3-bucket-3d364669-8ca1-50d5-9e1b-db8ba3fd4289 | 2024-10-02T04:25:57+07:00 | false | false > | aws-security-data-lake-ap-northeast-1-yd5aigzmgod7uadzgou2d2otb | arn:aws:s3:::aws-security-data-lake-ap-northeast-1-yd5aigzmgod7uadzgou2d2otb | 2022-12-01T17:07:10+07:00 | false | false > | sp-flow-s3bucket-logsink | arn:aws:s3:::sp-flow-s3bucket-logsink | 2022-07-27T19:53:02+07:00 ```
pdecat commented 3 weeks ago

Can confirm this resolves the NoSuchTagSet: The TagSet does not exist error I was facing with v1.2.0 of the plugin.