openedx / platform-plugin-aspects

Aspects plugins for the Open edX LMS and Studio
Apache License 2.0
2 stars 8 forks source link

DoesNotExist errors on (at least) ObjectTag Sink #88

Open bmtcril opened 3 weeks ago

bmtcril commented 3 weeks ago

Occasionally (2 out of 5 times for me) a model.DoesNotExist error will be thrown on (at least) ObjectTag sink.

platform_plugin_aspects.tasks.dump_data_to_clickhouse[32f2cd36-3aef-4771-8068-e5467dcb36a1] raised unexpected: DoesNotExist('ObjectTag matching query does not exist.')
Traceback (most recent call last):
  File "/openedx/venv/lib/python3.11/site-packages/celery/app/trace.py", line 453, in trace_task
    R = retval = fun(*args, **kwargs)
                 ^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/celery/app/trace.py", line 736, in __protected_call__
    return self.run(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/edx_django_utils/monitoring/internal/code_owner/utils.py", line 193, in new_function
    return wrapped_function(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/platform-plugin-aspects/platform_plugin_aspects/tasks.py", line 61, in dump_data_to_clickhouse
    sink.dump(object_id)
  File "/mnt/platform-plugin-aspects/platform_plugin_aspects/sinks/base_sink.py", line 191, in dump
    item = self.get_object(item_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^
  File "/mnt/platform-plugin-aspects/platform_plugin_aspects/sinks/base_sink.py", line 225, in get_object
    return self.get_model().objects.get(id=item_id)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/django/db/models/manager.py", line 87, in manager_method
    return getattr(self.get_queryset(), name)(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/openedx/venv/lib/python3.11/site-packages/django/db/models/query.py", line 637, in get
    raise self.model.DoesNotExist(
openedx_tagging.core.tagging.models.base.ObjectTag.DoesNotExist: ObjectTag matching query does not exist.

It's likely that we're getting to the Celery task before the MySQL transaction is committed in Studio. We should look at all of the tag models and see if any others might be prone to this. We have this pattern for working around the issue, and should get that fix in ASAP: https://github.com/openedx/platform-plugin-aspects/blob/e3fea7cf01767a51fcc6f57b8b29b1b63b5dcb91/platform_plugin_aspects/signals.py#L83

See also: https://github.com/openedx/platform-plugin-aspects/issues/54 , https://github.com/openedx/platform-plugin-aspects/pull/56/files