Feature idea: use a Cloudfront Origin Access Identity to protect S3

[pangolingo]

We could restrict access to the S3 bucket by using an origin access identity instead of a secret header. If there's any interest, I could try to get this working.

https://docs.aws.amazon.com/AmazonCloudFront/latest/DeveloperGuide/private-content-restricting-access-to-s3.html

[relnetops]

@ronaldtse This module is unusable due to security restrictions without support for OAI.

[kaihendry]

I did recently notice https://news.ycombinator.com/item?id=22350387

"This is supported with the website endpoint, but not with the REST endpoint that CloudFront uses when connecting it through the OAI." indicating to me that switching to OAI would be painful for anyone using redirects.

Would my routing rules still work if you made the switch? https://github.com/kaihendry/dabase.com/blob/master/terraform/s3.tf#L43

@relnetops What "security restrictions" are you referring to exactly?

[pangolingo]

Would my routing rules still work if you made the switch?

To the best of my knowledge, OAI is only an IAM rule forcing the end user to access the S3 bucket through Cloudfront. I wouldn't expect redirects to be affected. But it's Amazon, so who knows 🤷‍♂

If this repo implemented OAI, it could be an optional toggle - choose either OAI or the secret header.

[relnetops]

@kaihendry s3 buckets and their objects cannot be public. It's a major exfiltration vector.

[kaihendry]

@pangolingo happy :smile: to be the guinea pig and test your OAI changes

@relnetops I agree they probably shouldn't be public by default. Though sometimes people like to publicly share things.. like a Website. ;)

[ronaldtse]

Thank you @pangolingo @relnetops @kaihendry for the suggestion and comments!

We'd be more than happy to accept contributions on OAI, given the routing_rules can still be applied (I'm happy to test).

Do agree with @kaihendry on the public website comment :wink:

[ronaldtse]

I just checked documentation, and noticed this:

If you use an Amazon S3 bucket configured as a website endpoint, you must set it up with CloudFront as a custom origin and you can't use the origin access identity feature described in this topic. However, you can restrict access to content on a custom origin by setting up custom headers and configuring your origin to require them. For more information, see Restricting Access to Files on Custom Origins.

Maybe this was why we didn't implement it back then. Does this still apply?

[kaihendry]

With OAI, I believe you give up indexes, like index.html / error.html. This strikes as crazy. Especially when AWS propose getting them back with a Lambda@Edge in an incredibly convoluted manner: https://aws.amazon.com/blogs/compute/implementing-default-directory-indexes-in-amazon-s3-backed-amazon-cloudfront-origins-using-lambdaedge/

[ronaldtse]

Thanks @kaihendry for doing the research! I think it is reasonably straightforward to implement but this means every S3 bucket will be accompanied by a lambda function. Maybe that's acceptable for a watertight configuration.

The complication is if you need to attach an additional lambda authentication function like this, that authentication lambda function also needs to handle indexes. Which is actually not that bad.

This also means that we no longer need S3 acting as a website because the HTTP portion is entirely handled by Lambda.

@phuonghuynh what do you think?