Using django 4.2 storages

[alex-polosky]

Django recently released version 4.2, and with that version came a new settings / storage handler:

https://docs.djangoproject.com/en/4.2/ref/settings/#storages

https://docs.djangoproject.com/en/4.2/ref/files/storage/#django.core.files.storage.get_storage_class

I know that for messages in pyas2 the FileField is utilized, making it very easy to drop in AWS or GCS support. I was wondering if there are any plans to add support for the new storages setting / field option?

I believe the only change required would be to modify pyas2.model.Message to the following (or something similar):

class Message(...):
    ...
    def _get_storage(self):
        # Edited this to the correct code
        from django.core.files.storage import storages, InvalidStorageError
        try:
            return storages['pyas2']
        except InvalidStorageError:
            return storages['default']
    headers = ...(, storage=self._get_storage)
    payload = ...(, storage=self._get_storage)
    ...

Extra kwargs should be ignored, so this should be backwards compatible.

ETA: turns out that the storage kwarg exists back to at least 2.2, so there wouldn't be a need to worry about backwards compatibility.

[alex-polosky]

I am currently patching this in my settings file as follows:

...
from django.db.models import FileField
__original_init = FileField.__init__

def __get_storage():
    from django.core.files.storage import storages, InvalidStorageError
    try:
        return storages['pyas2']
    except InvalidStorageError:
        return storages['default']

def __override_init(self: FileField, *args, **kwargs):
    if 'storage' not in kwargs:
        upload_to = kwargs.get('upload_to', None)
        if upload_to:
            # These are pulled from pyas2.models as functions, but we can't import them at this stage
            if upload_to.__name__ in ('get_message_store', 'get_mdn_store'):
                kwargs['storage'] = __get_storage

    __original_init(self, *args, **kwargs)

FileField.__init__ = __override_init

STORAGES = {
    'default': {
        'BACKEND': 'django.core.files.storage.FileSystemStorage',
    },
    'staticfiles': {
        'BACKEND': 'django.contrib.staticfiles.storage.StaticFilesStorage',
    },
    'pyas2': {
        'BACKEND': 'storages.backends.gcloud.GoogleCloudStorage'
    }
}
...

[abhishek-ram]

We can already make use of DEFAULT_FILE_STORAGE and point it to storages.backends.gcloud.GoogleCloudStorage. Why does this approach not work?

[alex-polosky]

If DEFAULT_FILE_STORAGE is set to google cloud storage, the management commands will look to the google cloud storage for messages to send. IE: python manage.py sendas2message A B /path/to/file will check the google cloud storage bucket for the file, rather than the one on file storage.

Even if that is not an issue, according to the docs DEFAULT_FILE_STORAGE is now deprecated in favor of STORAGES.

[alex-polosky]

Also, looks like the code in MessageManager.create_from_as2message will need to be updated as it's using the default_storage call as well.