Compressor not compatible with Django Components?

[samuelesciancalepore]

I'm trying to implement Django Compressor according to Django Components, maybe I'm missing some points. Probably what I'm doing now is not enough.

When running the Django web application in a production like mode, using uvicorn and with DEBUG=0, the following error is reported when a request is made.

Full error message

Traceback (most recent call last):
  File ".../site-packages/django/core/handlers/exception.py", line 42, in inner
    response = await get_response(request)
  File ".../site-packages/django/utils/deprecation.py", line 150, in __acall__
    response = response or await self.get_response(request)
  File ".../site-packages/django/core/handlers/exception.py", line 44, in inner
    response = await sync_to_async(
  File ".../site-packages/asgiref/sync.py", line 468, in __call__
    ret = await asyncio.shield(exec_coro)
  File ".../lib/python3.10/concurrent/futures/thread.py", line 58, in run
    result = self.fn(*self.args, **self.kwargs)
  File ".../site-packages/asgiref/sync.py", line 520, in thread_handler
    return func(*args, **kwargs)
  File ".../site-packages/django/core/handlers/exception.py", line 134, in response_for_exception
    response = get_exception_response(
  File ".../site-packages/django/core/handlers/exception.py", line 167, in get_exception_response
    response = handle_uncaught_exception(request, resolver, sys.exc_info())
  File ".../site-packages/django/core/handlers/exception.py", line 185, in handle_uncaught_exception
    return callback(request)
  File ".../site-packages/django/utils/decorators.py", line 188, in _view_wrapper
    result = _process_exception(request, e)
  File ".../site-packages/django/utils/decorators.py", line 186, in _view_wrapper
    response = view_func(request, *args, **kwargs)
  File ".../site-packages/django/views/defaults.py", line 99, in server_error
    return HttpResponseServerError(template.render())
  File ".../site-packages/django/template/backends/django.py", line 61, in render
    return self.template.render(context)
  File ".../site-packages/django/template/base.py", line 171, in render
    return self._render(context)
  File ".../site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File ".../site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../site-packages/django/template/base.py", line 1000, in 
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
  File ".../site-packages/django/template/loader_tags.py", line 210, in render
    return template.render(context)
  File ".../site-packages/django/template/base.py", line 173, in render
    return self._render(context)
  File ".../site-packages/django/template/base.py", line 163, in _render
    return self.nodelist.render(context)
  File ".../site-packages/django/template/base.py", line 1000, in render
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../site-packages/django/template/base.py", line 1000, in 
    return SafeString("".join([node.render_annotated(context) for node in self]))
  File ".../site-packages/django/template/base.py", line 961, in render_annotated
    return self.render(context)
  File ".../site-packages/compressor/templatetags/compress.py", line 160, in render
    return self.render_compressed(
  File ".../site-packages/compressor/templatetags/compress.py", line 106, in render_compressed
    return self.render_offline(context)
  File ".../site-packages/compressor/templatetags/compress.py", line 84, in render_offline
    raise OfflineGenerationError(
compressor.exceptions.OfflineGenerationError: You have offline compression enabled but key "43914119d575b2bc3251b7078c6ccaf712d687b51d25588ef56a94ab60465d8e" is missing from offline manifest. You may need to run "python manage.py compress". Here is the original content:

<link type="text/x-scss" href="/static/css/app.scss" rel="stylesheet" media="screen">
<link type="text/css" href="/static/css/flatpickr/flatpickr.css" rel="stylesheet" media="screen">
<link href="/static/expandable_menu_item/style.css" media="all" rel="stylesheet">
<link href="/static/page_title/style.css" media="all" rel="stylesheet">
<link href="/static/sub_menu_item/style.css" media="all" rel="stylesheet">
<link href="/static/menu_group/style.css" media="all" rel="stylesheet">
<link href="/static/theme_switcher/style.css" media="all" rel="stylesheet">
<link href="/static/menu_item/style.css" media="all" rel="stylesheet">
<link href="/static/sidebar/style.css" media="all" rel="stylesheet">
<link href="/static/breadcrumbs/style.css" media="all" rel="stylesheet">
<link href="/static/avatar/style.css" media="all" rel="stylesheet">
<link href="/static/date_picker/style.css" media="all" rel="stylesheet">
<link href="/static/fullscreen_switcher/style.css" media="all" rel="stylesheet">
<link type="text/css" href="/static/css/ag-grid/ag-grid.css" rel="stylesheet" media="screen">
<link type="text/css" href="/static/css/ag-grid/ag-theme-quartz.css" rel="stylesheet" media="screen">

Research into the error reveals that django-compressor doesn't work as expected with django-components when using the component_css_dependencies and component_js_dependencies within a compress block.

It seems that these issues stem from the dynamic nature of the content generated within the compress block, which is used to generate the key (found in ./app/static/cache/manifest.json generated when running the compress command), results in a different key per request. But it's only a supposition, i'm not quite sure.

---

What has been implemented?

Implementation head-css.html

{% load compress %}
{% load static %}

{% compress css %}
    
    {% component_css_dependencies %}
{% endcompress %}

Implementation head-js.html

{% load compress %}
{% load static %}
{% load django_htmx %}

{% compress js %}
    {% component_js_dependencies %}
    
    
{% endcompress %}

Packages version

name = "django-components"
version = "0.95"

name = "django"
version = "5.1"

name = "django-compressor"
version = "4.5.1"

Does anyone have any idea why this isn't working well?

Thank you in advance

[EmilStenstrom]

@samuelesciancalepore Hi! I'm afraid we're not experts in django-compressor, and we currently don't have a working demo of it to play with either. If you want to play with the code and see if you can work around it, please feel free to post it as a PR afterwards, we'll happily accept contributions.

After talking to ChatGPT, it looks like the core of the problem is that django-compressor expects static content. But component_css_dependencies isn't static, it changed depending on which components are loaded. Perhaps there could be a workaround where you always render all dependencies and cache them?

[JuroOravec]

Same as @EmilStenstrom, I'm not familiar with django-compressor.

Based on Emil's ChatGPT convo, superficially, it sounds to me like the ideal solution would be if django-compressor made it possible to compress both dynamic and static assets in a single template, e.g. by adding an optional dynamic kwarg to their {% compress %} template tag:

<head>
  {# static assets, would be extracted with offline compression #}
  {% compress js %}
    <script src="/static/path/to/script.js"></script>
    <link href="/static/path/to/style.css">
  {% endcompress %}

  {# dynamic assets would NOT be extracted with offline compression #}
  {% compress js dynamic %}
    {% component_js_dependencies %}
  {% endcompress %}
</head>

---

So @samuelesciancalepore I'd suggest to cross-post this issue on django-compressor.

---

Compatibility with django-compressor

Maybe this is a good place to discuss compatibility with django-compressor.

Assuming that django-compressor implemented the dynamic kwarg, I think there would still be some work on our side to make it compatible.

For context, we're currently reworking how django-components handles the JS / CSS assets.

But even after that refactor, the outputs of {% component_dependencies %} would be dynamic, depending on which components were used in the overall HTML doc that was rendered.

But to minimize how much compressing has to be done at runtime, we could add kwargs dynamic and static to output only dynamic / static assets (these would be mutually exclusive):

<head>
  {# static assets, would be extracted with offline compression #}
  {% compress js %}
    <script src="/static/path/to/script.js"></script>
    <link href="/static/path/to/style.css">
    {% component_js_dependencies static %}
  {% endcompress %}

  {# dynamic assets would NOT be extracted with offline compression #}
  {% compress js dynamic %}
    {% component_js_dependencies dynamic %}
  {% endcompress %}
</head>

The normal {% component_dependencies %} without dynamic nor static would render all (as it is now).

Moreover, down the line, I'd like to add a support for TS or Sass for the inlined code. This would involve extracting out the TS / Sass code and compiling it into JS / CSS static files. So I have a hunch that being able to selectively render only dynamic or only static assets with {% component_dependencies %} would be even more valuable for that.

PS: I don't know much about how django-compressor works internally, so pls correct me if I got something wrong.