search

jschneier / django-storages

https://django-storages.readthedocs.io/
BSD 3-Clause "New" or "Revised" License
2.7k stars 847 forks source link

"ValueError" with staticfiles configured to use the Azure Storage #1391

Open The-Judge opened 2 months ago

The-Judge commented 2 months ago

Following the manual to use django-storages with Azure and staticfiles storage, I end up with this Django configuration:

STORAGES["staticfiles"] = {
            "BACKEND": "storages.backends.azure_storage.AzureStorage",
            "OPTIONS": {
                "account_name": os.getenv("AZURE_SA_NAME", None),
                "azure_container": os.getenv("AZURE_CONTAINER_NAME_STATIC", "static"),
                "overwrite_files": True if os.getenv("AZURE_OVERWRITE_FILES", "False").capitalize() == "True" else False,
                "account_key": os.getenv("AZURE_ACCOUNT_KEY", None),
            }
        }

Running the manage.py task collectstatic works fine and stores the static files in the Azure Storage Account container static. When accessing the Page, this is the issue:

Internal Server Error: /
Traceback (most recent call last):
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/core/handlers/exception.py", line 55, in inner
    response = get_response(request)
               ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/core/handlers/base.py", line 220, in _get_response
    response = response.render()
               ^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/sentry_sdk/integrations/django/views.py", line 39, in sentry_patched_render
    return old_render(self)
           ^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/response.py", line 114, in render
    self.content = self.rendered_content
                   ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/sentry_sdk/utils.py", line 1711, in runner
    return sentry_patched_function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/sentry_sdk/integrations/django/templates.py", line 76, in rendered_content
    return real_rendered_content.fget(self)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/response.py", line 92, in rendered_content
    return template.render(context, self._request)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/base.py", line 171, in render
    return self._render(context)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/loader_tags.py", line 159, in render
    return compiled_parent._render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/test/utils.py", line 111, in instrumented_test_render
    return self.nodelist.render(context)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
                              ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/base.py", line 1000, in <listcomp>
    return SafeString("".join([node.render_annotated(context) for node in self]))
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
           ^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/templatetags/static.py", line 116, in render
    url = self.url(context)
          ^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/templatetags/static.py", line 113, in url
    return self.handle_simple(path)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/django/templatetags/static.py", line 129, in handle_simple
    return staticfiles_storage.url(path)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/debug_toolbar/panels/staticfiles.py", line 64, in url
    return super().url(path)
           ^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/storages/backends/azure_storage.py", line 332, in url
    container_blob_url = self.custom_client.get_blob_client(name).url
                         ^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/storages/backends/azure_storage.py", line 212, in custom_client
    self._custom_client = self.custom_service_client.get_container_client(
                          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/azure/storage/blob/_blob_service_client.py", line 735, in get_container_client
    return ContainerClient(
           ^^^^^^^^^^^^^^^^
  File "/Users/marichter/.local/share/virtualenvs/dc-django-Jq1XImY9/lib/python3.11/site-packages/azure/storage/blob/_container_client.py", line 158, in __init__
    raise ValueError("Please specify a container name.")
ValueError: Please specify a container name.
jschneier commented 2 months ago

Thanks for the report, a number of people have reported similar issues across all backends and I'm simply unable to reproduce the issue. Do you have a minimal example?

Can you share a few things:

  1. Django version
  2. Any other packages you use in your pipeline that impact static files
  3. A list of all storages settings (obviously sanitized)
The-Judge commented 2 months ago

I just tried to reproduce this with the following steps:

  1. Create Storage Account: az storage account create --name <some-unique-name> --resource-group <some-resource-group> --subscription <your-subscription-id> --public-network-access Enabled --allow-blob-public-access true
  2. Get a Key for the SA: az storage account keys list --account-name <some-unique-name> --resource-group <some-resource-group> --subscription <your-subscription-id>. Take note of any of those two values.
  3. Create a container for the static data: az storage container create --name static --subscription <your-subscription-id> --account-name <some-unique-name> --account-key '<key-from-step-2>' --public-access container
  4. Create a new folder somewhere to hold the demo-django.
  5. Create following Pipfile within: 
    
[[source]]
url = "https://pypi.org/simple"
verify_ssl = true
name = "pypi"

[pipenv] allow_prereleases = false disable_pip_input = true

[packages] django = "~=5.0.4" whitenoise = {extras = ["brotli"], version = ""} gunicorn = "" dj_database_url = "" lz4 = "" django-storages = {extras = ["azure"], version = "*"}

[dev-packages]

[requires] python_version = "3.11"

6. Create Pipenv venv: `pipenv update --dev`
7. Activate venv: `source $(pipenv --venv)/bin/activate`
8. Start Django Project: `django-admin startproject demoproject`
9. In `settings.py`, add `"whitenoise.runserver_nostatic",` above `"django.contrib.staticfiles",` and `"storages",` below to `INSTALLED_APPS`:
```python
INSTALLED_APPS = [
    "django.contrib.admin",
    "django.contrib.auth",
    "django.contrib.contenttypes",
    "django.contrib.sessions",
    "django.contrib.messages",
    "whitenoise.runserver_nostatic",
    "django.contrib.staticfiles",
    "storages",
]
  1. In settings.py, add STORAGES: 
    STORAGES = {
"default": {
    "BACKEND": "django.core.files.storage.FileSystemStorage",
},
"staticfiles": {
        "BACKEND": "storages.backends.azure_storage.AzureStorage",
        "OPTIONS": {
            "account_name": 'MY_SA_NAME',
            "azure_container": 'static',
            "overwrite_files": True,
            "account_key": 'MY_SA_KEY',
        }
}
}

"Unfortunately", I couldn't; this, way everything seemed to work. I will try some more to find the reason this happens in my Wagtail based project.