Incompatibility with latest Django 3.2.19 security release

[sbesson]

See https://www.djangoproject.com/weblog/2023/may/03/security-releases/

Django 3.2.19 disabled the ability to upload multiple files using one form field (see https://github.com/django/django/commit/eed53d0011622e70b936e203005f0e6f4ac48965). Unfortunately, we started consuming this mechanism in https://github.com/ome/omero-web/pull/410 to allow multiple file annotations to be uploaded via the UI.

Currently if Django is installed or upgraded to 3.2.19, OMERO.web 5.17+ will fail to start with the following exception

ValueError: ClearableFileInput doesn't support uploading multiple files.

As an immediate workaround, capping Django to 3.2.18 should restore the ability to start OMERO.web. This issue should likely be addressed properly in a patch release of OMERO.web restoring compatibility with the Django 3.2.x line.

My inclination would be to disable the feature introduced in #410 for now and look into reintroducing it at a later date using a supported API.

/cc @knabar @chris-allan @will-moore @pwalczysko @jburel

[will-moore]

Looks like the MultipleFileInput subclass in https://github.com/django/django/commit/eed53d0011622e70b936e203005f0e6f4ac48965 is worth trying (and preferable to removing the feature)?

[chris-allan]

Possibly, though I don't think it's that simple due to potential expected "cleaning" that is currently being done by omeroweb.custom_forms.NonASCIIForm. However, a lot of the code there is very old so who knows if it's even required anymore.

[wheresvic]

Unfortunately this problem is manifesting itself on a fresh omero web install and leads to the following error when trying to start the server (or manually do omero web clearsessions):

Clearing expired sessions. This may take some time... Traceback (most recent call last):
  File "manage.py", line 75, in <module>
    execute_from_command_line(sys.argv)
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/core/management/__init__.py", line 419, in execute_from_command_line
    utility.execute()
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/core/management/__init__.py", line 413, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/core/management/base.py", line 354, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/core/management/base.py", line 393, in execute
    self.check()
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/core/management/base.py", line 419, in check
    all_issues = checks.run_checks(
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/core/checks/registry.py", line 76, in run_checks
    new_errors = check(app_configs=app_configs, databases=databases)
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/core/checks/urls.py", line 13, in check_url_config
    return check_resolver(resolver)
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/core/checks/urls.py", line 23, in check_resolver
    return check_method()
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/urls/resolvers.py", line 416, in check
    for pattern in self.url_patterns:
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/urls/resolvers.py", line 602, in url_patterns
    patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/utils/functional.py", line 48, in __get__
    res = instance.__dict__[self.name] = self.func(instance)
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/urls/resolvers.py", line 595, in urlconf_module
    return import_module(self.urlconf_name)
  File "/usr/lib/python3.8/importlib/__init__.py", line 127, in import_module
    return _bootstrap._gcd_import(name[level:], package, level)
  File "<frozen importlib._bootstrap>", line 1014, in _gcd_import
  File "<frozen importlib._bootstrap>", line 991, in _find_and_load
  File "<frozen importlib._bootstrap>", line 975, in _find_and_load_unlocked
  File "<frozen importlib._bootstrap>", line 671, in _load_unlocked
  File "<frozen importlib._bootstrap_external>", line 848, in exec_module
  File "<frozen importlib._bootstrap>", line 219, in _call_with_frames_removed
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/omeroweb/urls.py", line 38, in <module>
    from omeroweb.webclient import views as webclient_views
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/omeroweb/webclient/views.py", line 73, in <module>
    from .forms import GlobalSearchForm, ContainerForm
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/omeroweb/webclient/forms.py", line 328, in <module>
    class FilesAnnotationForm(BaseAnnotationForm):
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/omeroweb/webclient/forms.py", line 338, in FilesAnnotationForm
    widget=forms.ClearableFileInput(attrs={"multiple": True}), required=False
  File "/opt/omero/web/venv3/lib/python3.8/site-packages/django/forms/widgets.py", line 391, in __init__
    raise ValueError(
ValueError: ClearableFileInput doesn't support uploading multiple files.

Currently I just fixed it by manually editing /opt/omero/web/venv3/lib/python3.8/site-packages/omeroweb/webclient/forms.py, line 338:

# widget=forms.ClearableFileInput(attrs={"multiple": True}), required=False
widget=forms.ClearableFileInput(), required=False

With this I could at least start omero. I hope it does not break any functionality down the road but yolo.

[chris-allan]

The easiest option until this issue is resolved is to temporarily downgrade your Django version to 3.2.18.

[wheresvic]

@chris-allan I'm a bit of a noob with python. How would I downgrade django?

[sbesson]

@wheresvic pip install Django==3.2.18

[wheresvic]

It took a bit of trial and error but thank you very much @sbesson and @chris-allan 👍🏾

Run as root (as per the omero web installation instructions): sudo /opt/omero/web/venv3/bin/pip install Django==3.2.18

does the trick :)

[joshmoore]

Thanks @wheresvic.

For anyone finding this, please see also:

• https://www.openmicroscopy.org/security/advisories/2023-31047/
• https://forum.image.sc/t/omero-web-affected-by-django-3-2-19-security-release/80810