search

tfranzel / drf-spectacular

Sane and flexible OpenAPI 3 schema generation for Django REST framework.
https://drf-spectacular.readthedocs.io
BSD 3-Clause "New" or "Revised" License
2.4k stars 266 forks source link

Bug: ERROR Exception inside application: __provides__ #1296

Open deirdreamuel opened 1 month ago

deirdreamuel commented 1 month ago

Describe the bug There is an issue with drf_spectacular's @extend_schema_view that is only happening in production server i.e. Daphne, Gunicorn, Uvicorn. This issue does not happen with python manage.py runserver. The server does start up with daphne after removing all @extend_schema_view decorators.

Sample logs using Daphne:


2024-09-18 03:38:47,738 ERROR    Exception inside application: __provides__
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/asgiref/sync.py", line 518, in thread_handler
raise exc_info[1]
File "/usr/local/lib/python3.12/site-packages/django/core/handlers/exception.py", line 42, in inner
response = await get_response(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/handlers/base.py", line 235, in _get_response_async
callback, callback_args, callback_kwargs = self.resolve_request(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/handlers/base.py", line 313, in resolve_request
resolver_match = resolver.resolve(request.path_info)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/resolvers.py", line 686, in resolve
for pattern in self.url_patterns:
^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/utils/functional.py", line 47, in __get__
res = instance.__dict__[self.name] = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/resolvers.py", line 738, in url_patterns
patterns = getattr(self.urlconf_module, "urlpatterns", self.urlconf_module)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/utils/functional.py", line 47, in __get__
res = instance.__dict__[self.name] = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/resolvers.py", line 731, in urlconf_module
return import_module(self.urlconf_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/usr/src/api_server/app/urls.py", line 14, in <module>
path("v1.0/", include("core.urls")),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/conf.py", line 39, in include
urlconf_module = import_module(urlconf_module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/usr/src/api_server/core/urls/__init__.py", line 3, in <module>
from core.urls import (
File "/usr/src/api_server/core/urls/organization.py", line 3, in <module>
from core.views import organization
File "/usr/src/api_server/core/views/organization.py", line 17, in <module>
@extend_schema_view(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/drf_spectacular/utils.py", line 658, in decorator
available_view_methods = get_view_method_names(view)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/drf_spectacular/drainage.py", line 183, in get_view_method_names
item for item in dir(view) if callable(getattr(view, item)) and (
^^^^^^^^^^^^^^^^^^^
AttributeError: __provides__. Did you mean: '__providedBy__'?
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
File "/usr/local/lib/python3.12/site-packages/asgiref/sync.py", line 518, in thread_handler
raise exc_info[1]
File "/usr/local/lib/python3.12/site-packages/django/core/handlers/exception.py", line 42, in inner
response = await get_response(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/utils/deprecation.py", line 150, in __acall__
response = response or await self.get_response(request)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/handlers/exception.py", line 44, in inner
response = await sync_to_async(
^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/asgiref/sync.py", line 468, in __call__
ret = await asyncio.shield(exec_coro)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/asgiref/sync.py", line 520, in thread_handler
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/handlers/exception.py", line 140, in response_for_exception
response = handle_uncaught_exception(
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/core/handlers/exception.py", line 184, in handle_uncaught_exception
callback = resolver.resolve_error_handler(500)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/resolvers.py", line 752, in resolve_error_handler
callback = getattr(self.urlconf_module, "handler%s" % view_type, None)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/utils/functional.py", line 47, in __get__
res = instance.__dict__[self.name] = self.func(instance)
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/resolvers.py", line 731, in urlconf_module
return import_module(self.urlconf_name)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/usr/src/api_server/app/urls.py", line 14, in <module>
path("v1.0/", include("core.urls")),
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/django/urls/conf.py", line 39, in include
urlconf_module = import_module(urlconf_module)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/importlib/__init__.py", line 90, in import_module
return _bootstrap._gcd_import(name[level:], package, level)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "<frozen importlib._bootstrap>", line 1387, in _gcd_import
File "<frozen importlib._bootstrap>", line 1360, in _find_and_load
File "<frozen importlib._bootstrap>", line 1331, in _find_and_load_unlocked
File "<frozen importlib._bootstrap>", line 935, in _load_unlocked
File "<frozen importlib._bootstrap_external>", line 995, in exec_module
File "<frozen importlib._bootstrap>", line 488, in _call_with_frames_removed
File "/usr/src/api_server/core/urls/__init__.py", line 3, in <module>
from core.urls import (
File "/usr/src/api_server/core/urls/organization.py", line 3, in <module>
from core.views import organization
File "/usr/src/api_server/core/views/organization.py", line 17, in <module>
@extend_schema_view(
^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/drf_spectacular/utils.py", line 658, in decorator
available_view_methods = get_view_method_names(view)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/usr/local/lib/python3.12/site-packages/drf_spectacular/drainage.py", line 183, in get_view_method_names
item for item in dir(view) if callable(getattr(view, item)) and (
^^^^^^^^^^^^^^^^^^^
AttributeError: __provides__. Did you mean: '__providedBy__'?

To Reproduce I have something sort of the following for different views under /views dir.

@extend_schema_view(
    list=extend_schema(
        tags=["Task Action"],
        operation_id="Task Action List",
        description="Retrieves a list of all task actions that can be applied.",
    ),
    retrieve=extend_schema(
        tags=["Task Action"],
        operation_id="Task Action Retrieve",
        description="Retrieves a task action data by Public ID.",
    ),
)

Expected behavior The application should work and start without problems the same way it starts with python manage.py runserver

tfranzel commented 1 month ago

Interesting! So it looks like getattr(view, item) fails even though it should by definition succeed. dir(view) gets all the attribute names right there and thus they should alle be getattr-able, unless you have a custom __dir__ implementation on the the view.

However, since uvicorn et al do not even remotely go there, I cannot really understand why your dev/prod behaves differently, unless there is other things going on, you are not realizing.

A quick google search on __provides__ only gave some results on zope. Are you using that? Apart from that, this does not seem to be a common dunder method.

https://github.com/tfranzel/drf-spectacular/blob/f90e7bf96b8a96c1515116011836b9a91e0dd30e/drf_spectacular/drainage.py#L182-L189

sshishov commented 1 month ago

@tfranzel please look into https://github.com/twisted/twisted codebase. We are having the same issue and can be backtracked there: https://github.com/scrapy/scrapy/issues/6307#issuecomment-2042341378

Related issues:

I assume we should apply the same fix, do not rely that all methods/attributes returned by dir are available always, mentioned here: https://docs.python.org/3.10/library/functions.html#dir