anomaly / lab-python-server

A template for building containerised web applications in Python with a recommended set o f dependencies and tooling
Apache License 2.0
11 stars 2 forks source link

Provide helper methods to use `boto3` instead of the `minio` client #82

Open devraj opened 1 year ago

devraj commented 1 year ago

Is your feature request related to a problem? Please describe. While working on an AWS project using this stack, we realised that we have to use boto3 for various things like accessing the SecretsManager and objects in s3 when the environment exchanges credentials using IAM policies (where the application itself does not need the keys to access the bucket).

It would be nice to provide a boto3 wrapped up to work with this stack.

Describe the solution you'd like A set of wrapper methods on the s3_file_metadata table that allow us to work with IAM managed s3 buckets. These can be specifically be prefixed with aws_ to indicate that they work in an AWS managed environment only.

Secondly consider best practices around working with multiple buckets, in our use case we use a bucket to provide media files (accessed via a CDN) and the the other has user generated content which are only to be accessed via signed urls.

As for documentation it will be helpful to provide assumptions and samples.

Describe alternatives you've considered NA

Additional context See aws-eks-cluster for a terraform deployed AWS environment which is setup in the way described above.

devraj commented 1 year ago

See also #19