unity-sds / unity-sps

The Unity SDS Processing Service facilitates large-scale data processing for scientific workflows.
Apache License 2.0
2 stars 2 forks source link

[New Feature]: Enforce SSL access on SPS S3 buckets #231

Open LucaCinquini opened 3 weeks ago

LucaCinquini commented 3 weeks ago

MCP is now mandating that all access to S3 buckets use SSL.

See MCP ticket: https://jaas.gsfc.nasa.gov/servicedesk/customer/portal/2/GSD-4081

See remediation steps: https://docs.aws.amazon.com/securityhub/latest/userguide/s3-controls.html#s3-5

It seems like SPS creates 4 S3 buckets for each deployment, for example:

We must update the Terraform script to add the referenced policy to each of these buckets, and then redeploy all instances (or manually attach the policy if a redeployment is not desired).

Currently there are 39 non-compliant SPS S3 buckets.

LucaCinquini commented 3 days ago

@jpl-btlunsfo : after Galen deleted some S3 buckets, I get a different error:

module.unity-sps-initiators.module.unity_initiator.aws_lambda_event_source_mapping.initiator_queue_event_source_mapping: Creation complete after 25s [id=4b06890d-d8ea-48b3-b1e1-d7661c79398f] ╷ │ Error: putting S3 Bucket (unity-luca-1-dev-sps-unity-luca-1-dev-sps-isl) Policy: operation error S3: PutBucketPolicy, https response error StatusCode: 404, RequestID: 30FPN54YSAT2V6M5, HostID: KkErD2hE7gmtDEkwE/kIP0oILH7B4B3gYXFUNBPUftMXbMLlnA+J121tnBrsYLffPRNSg1eh1RU=, api error NoSuchBucket: The specified bucket does not exist │ │ with module.unity-sps-initiators.aws_s3_bucket_policy.ssl_s3_policy["isl"], │ on modules/terraform-unity-sps-initiators/main.tf line 31, in resource "aws_s3_bucket_policy" "ssl_s3_policy": │ 31: resource "aws_s3_bucket_policy" "ssl_s3_policy" { │ ╵ ╷ │ Error: putting S3 Bucket (unity-luca-1-dev-sps-unity-luca-1-dev-sps-config) Policy: operation error S3: PutBucketPolicy, https response error StatusCode: 404, RequestID: 30FGHGFXX74PG5P6, HostID: 59W2+tt8U9Oc72o1l5P52NGEOBECGcsF2PSKkPe96ILBJYYJq7jg5v81xZOuI+8ZhQCClCySAdA=, api error NoSuchBucket: The specified bucket does not exist │ │ with module.unity-sps-initiators.aws_s3_bucket_policy.ssl_s3_policy["config"], │ on modules/terraform-unity-sps-initiators/main.tf line 31, in resource "aws_s3_bucket_policy" "ssl_s3_policy": │ 31: resource "aws_s3_bucket_policy" "ssl_s3_policy" { │ ╵ ╷ │ Error: putting S3 Bucket (unity-luca-1-dev-sps-unity-luca-1-dev-sps-code) Policy: operation error S3: PutBucketPolicy, https response error StatusCode: 404, RequestID: 30FGNH4RFJJW9X9W, HostID: dqLAYFGhOAknX4WRENiSjjy67txRWd/VsB8ZEXyyt+jtf4+sw2fRH7Nx18lGsu6jKDC6eBy/Anc=, api error NoSuchBucket: The specified bucket does not exist │ │ with module.unity-sps-initiators.aws_s3_bucket_policy.ssl_s3_policy["code"], │ on modules/terraform-unity-sps-initiators/main.tf line 31, in resource "aws_s3_bucket_policy" "ssl_s3_policy": │ 31: resource "aws_s3_bucket_policy" "ssl_s3_policy" {

Seems like there is some naming repetition?

LucaCinquini commented 1 day ago

See comment on PR about changing one line of code.