jschneier / django-storages

https://django-storages.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
2.7k stars 847 forks source link

using S3 access points #1415

Open plenaerts opened 3 weeks ago

plenaerts commented 3 weeks ago

How should I configure django-storages for static and media files while using an S3 access point? I want to do this because I run django on an AWS AppRunner service, which connects via a private VPC to access points for RDS, S3, other AWS services.

Right now I have the config below in my settings. I replaced them with placeholders here, but in my deploy I read them from environment or AWS secrets.

The result of this config is that:

  1. The value in bucket_name is used as a folder on my S3 bucket. I can put whatever value in bucket_name if it passes bucket_name validation. The access point URL in endpoint_url already points to the right bucket and I'm stuck with an extra folder in my bucket.
  2. The value in custom_domain needs to be appended with the value in bucket_name as that became a folder in the bucket. So that's my cloudfront custom domain appended with '/bucket_name'.

It works, but my folder structure in my bucket is not what I intended and it took some time trying to figure out how to use my access point.

Is this how django-storages is supposed to work with S3 access points? If so, then an example in the documentation would be useful I think.

AWS_S3_ACCESS_KEY_ID = my_key_id
AWS_S3_SECRET_ACCESS_KEY = my_secret_key
AWS_S3_REGION_NAME = my_region
AWS_S3_BUCKET_NAME = any_value_not_even_the_bucket_name
AWS_S3_CUSTOM_DOMAIN = my_cloudfront_domain/bucket_name
AWS_S3_ACCOUNT_ID = my_account_id
AWS_S3_ACCESS_POINT_NAME = my_access_point_name

STORAGES = {
    "default": {
        "BACKEND": "storages.backends.s3.S3Storage",
        "OPTIONS": {
            'access_key': AWS_S3_ACCESS_KEY_ID,
            'secret_key': AWS_S3_SECRET_ACCESS_KEY,
            'bucket_name': AWS_S3_BUCKET_NAME,
            'region_name': AWS_S3_REGION_NAME,
            'file_overwrite': True,
            'location': 'media',
            'default_acl': 'public-read',
            'custom_domain': AWS_S3_CUSTOM_DOMAIN,
            'endpoint_url': f'https://{AWS_S3_ACCESS_POINT_NAME}-{AWS_S3_ACCOUNT_ID}.s3-accesspoint.{AWS_S3_REGION_NAME}.amazonaws.com',
        },
    },
    "staticfiles": {
        "BACKEND": "storages.backends.s3.S3Storage",
        "OPTIONS": {
            'access_key': AWS_S3_ACCESS_KEY_ID,
            'secret_key': AWS_S3_SECRET_ACCESS_KEY,
            'bucket_name': AWS_S3_BUCKET_NAME,
            'region_name': AWS_S3_REGION_NAME,
            'file_overwrite': True,
            'default_acl': 'public-read',
            'location': 'static',
            'querystring_auth': False,
            'custom_domain': AWS_S3_CUSTOM_DOMAIN,
            'endpoint_url': f'https://{AWS_S3_ACCESS_POINT_NAME}-{AWS_S3_ACCOUNT_ID}.s3-accesspoint.{AWS_S3_REGION_NAME}.amazonaws.com',
        },
    },
}