search

pulp / pulpcore

Pulp 3 pulpcore package https://pypi.org/project/pulpcore/
GNU General Public License v2.0
284 stars 111 forks source link

race condition when deleting a repository and associated distribution #2019

Open pulpbot opened 2 years ago

pulpbot commented 2 years ago

Author: @jlsherrill (jsherril@redhat.com)

Redmine Issue: 8942, https://pulp.plan.io/issues/8942

In our katello tests, we have some 'cleanup' code that:

list and then deletes a repository with a given name list and then deletes a remote with a given name list and then deletes a distribution with a given name wait on all the tasks generated from above

This has worked for quite a while, but after upgrading from pulp-container 2.3 to 2.6, we started seeing an error (ISE) from the server:

LookupError: view not found

when trying to list the distributions on the 3rd step.

I'm not sure if the pulp code is running faster, or slower, but it appears that there is a consistent race condition occuring. I can modify the code to poll the tasks after the repo delete, and it seems to work find (so this isn't blocking us)

Here's the full traceback:


Untitled
From WebUI, 1 Second ago, written in Plain Text.
This paste will self destruct in 1 Day. 
URL https://paste.centos.org/view/99464b52
Embed Show code
Download Paste or View Raw

        Jun 21 16:14:09 devel2 pulpcore-api: pulp [b51142c5bced40989a59403cf6e92fd8]: django.request:ERROR: Internal Server Error: /pulp/api/v3/distributions/container/container/
        Jun 21 16:14:09 devel2 pulpcore-api: Traceback (most recent call last):
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/django/core/handlers/exception.py", line 34, in inner
        Jun 21 16:14:09 devel2 pulpcore-api: response = get_response(request)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 115, in _get_response
        Jun 21 16:14:09 devel2 pulpcore-api: response = self.process_exception_by_middleware(e, request)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/django/core/handlers/base.py", line 113, in _get_response
        Jun 21 16:14:09 devel2 pulpcore-api: response = wrapped_callback(request, *callback_args, **callback_kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
        Jun 21 16:14:09 devel2 pulpcore-api: return view_func(*args, **kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/viewsets.py", line 125, in view
        Jun 21 16:14:09 devel2 pulpcore-api: return self.dispatch(request, *args, **kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 509, in dispatch
        Jun 21 16:14:09 devel2 pulpcore-api: response = self.handle_exception(exc)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 469, in handle_exception
        Jun 21 16:14:09 devel2 pulpcore-api: self.raise_uncaught_exception(exc)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 480, in raise_uncaught_exception
        Jun 21 16:14:09 devel2 pulpcore-api: raise exc
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/views.py", line 506, in dispatch
        Jun 21 16:14:09 devel2 pulpcore-api: response = handler(request, *args, **kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/mixins.py", line 43, in list
        Jun 21 16:14:09 devel2 pulpcore-api: return self.get_paginated_response(serializer.data)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 745, in data
        Jun 21 16:14:09 devel2 pulpcore-api: ret = super().data
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 246, in data
        Jun 21 16:14:09 devel2 pulpcore-api: self._data = self.to_representation(self.instance)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 664, in to_representation
        Jun 21 16:14:09 devel2 pulpcore-api: self.child.to_representation(item) for item in iterable
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 664, in <listcomp>
        Jun 21 16:14:09 devel2 pulpcore-api: self.child.to_representation(item) for item in iterable
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/serializers.py", line 515, in to_representation
        Jun 21 16:14:09 devel2 pulpcore-api: ret[field.field_name] = field.to_representation(attribute)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/rest_framework/relations.py", line 401, in to_representation
        Jun 21 16:14:09 devel2 pulpcore-api: url = self.get_url(value, self.view_name, request, format)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/serializers/fields.py", line 210, in get_url
        Jun 21 16:14:09 devel2 pulpcore-api: repo_url = rvr_field.get_url(obj.repository, None, request, *args, **kwargs)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/serializers/fields.py", line 198, in get_url
        Jun 21 16:14:09 devel2 pulpcore-api: return super().get_url(obj, self.view_name, request, *args, **kwargs) + "versions/"
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/serializers/base.py", line 236, in get_url
        Jun 21 16:14:09 devel2 pulpcore-api: view_name = self._view_name(obj)
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/serializers/base.py", line 231, in _view_name
        Jun 21 16:14:09 devel2 pulpcore-api: return get_view_name_for_model(obj, "detail")
        Jun 21 16:14:09 devel2 pulpcore-api: File "/usr/lib/python3.6/site-packages/pulpcore/app/util.py", line 78, in get_view_name_for_model
        Jun 21 16:14:09 devel2 pulpcore-api: raise LookupError("view not found")
        Jun 21 16:14:09 devel2 pulpcore-api: LookupError: view not found
        Jun 21 16:14:09 devel2 pulpcore-api: pulp [b51142c5bced40989a59403cf6e92fd8]:  - - [21/Jun/2021:16:14:09 +0000] "GET /pulp/api/v3/distributions/container/container/?name=Default_Organization-Test-busybox-library HTTP/1.1" 500 27 "-" "OpenAPI-Generator/2.4.0/ruby"
stale[bot] commented 2 years ago

This issue has been marked 'stale' due to lack of recent activity. If there is no further activity, the issue will be closed in another 30 days. Thank you for your contribution!