django-filter: KeyError: 'required'

[jesusch]

while using django_mongoengine_filter.FilterSet I got the following error:

  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/django_filters/rest_framework/backends.py", line 139, in <listcomp>
    ) for field_name, field in filterset_class.base_filters.items()
KeyError: 'required'

problem: django-filter Filtter object (meanwhile?) relies on having a self.extra['required'] property see: https://github.com/carltongibson/django-filter/blob/da4b64ea8dde19304b552592a32b343e9aefe9da/django_filters/filters.py#L79

while generating OpenAPI schemas this property is checked

[jesusch]

when the constructor get's changed, also other places break e.g.: https://github.com/jesusch/django-mongoengine-filter/blob/master/django_mongoengine_filter/filters.py#L97

which results in TypeError: type object got multiple values for keyword argument 'required'

[barseghyanartur]

@jesusch:

Please, include the full trace and your software versions.

It does work as shown in the example. Please, include a full example of how you're using it.

[jesusch]

For testing purpose I meanwhile switched to the sync-api branch

software versions:

Package                           Version    Location                                                      
--------------------------------- ---------- --------------------------------------------------------------
certifi                           2020.4.5.1 
chardet                           3.0.4      
coreapi                           2.3.3      
coreschema                        0.0.4      
Django                            2.2.12     
django-cors-headers               3.3.0      
django-filter                     2.2.0      
django-mongoengine                0.4.1      
django-mongoengine-filter         0.3.5      /Users/jesusch/git/jobq-api/venv/src/django-mongoengine-filter
django-rest-framework-mongoengine 3.4.1      
djangorestframework               3.11.0     
drf-yasg                          1.17.1     
idna                              2.9        
inflection                        0.4.0      
itypes                            1.2.0      
Jinja2                            2.11.2     
MarkupSafe                        1.1.1      
mongoengine                       0.20.0     
packaging                         20.4       
pip                               19.2.3     
pymongo                           3.10.1     
pyparsing                         2.4.7      
pytz                              2020.1     
requests                          2.23.0     
ruamel.yaml                       0.16.10    
ruamel.yaml.clib                  0.2.0      
setuptools                        41.2.0     
six                               1.15.0     
sqlparse                          0.3.1      
uritemplate                       3.0.1      
urllib3                           1.25.9 

Full Traceback:

Traceback (most recent call last):
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 115, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/django/core/handlers/base.py", line 113, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/django/views/generic/base.py", line 71, in view
    return self.dispatch(request, *args, **kwargs)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/rest_framework/views.py", line 505, in dispatch
    response = self.handle_exception(exc)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/rest_framework/views.py", line 465, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/rest_framework/views.py", line 476, in raise_uncaught_exception
    raise exc
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/rest_framework/views.py", line 502, in dispatch
    response = handler(request, *args, **kwargs)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/drf_yasg/views.py", line 94, in get
    schema = generator.get_schema(request, self.public)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/drf_yasg/generators.py", line 254, in get_schema
    paths, prefix = self.get_paths(endpoints, components, request, public)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/drf_yasg/generators.py", line 412, in get_paths
    operation = self.get_operation(view, path, prefix, method, components, request)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/drf_yasg/generators.py", line 454, in get_operation
    operation = view_inspector.get_operation(operation_keys)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/drf_yasg/inspectors/view.py", line 33, in get_operation
    query = self.get_query_parameters()
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/drf_yasg/inspectors/view.py", line 291, in get_query_parameters
    natural_parameters = self.get_filter_parameters() + self.get_pagination_parameters()
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/drf_yasg/inspectors/base.py", line 416, in get_filter_parameters
    fields += self.probe_inspectors(self.filter_inspectors, 'get_filter_parameters', filter_backend()) or []
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/drf_yasg/inspectors/base.py", line 118, in probe_inspectors
    result = method(obj, **kwargs)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/drf_yasg/inspectors/query.py", line 26, in get_filter_parameters
    fields = filter_backend.get_schema_fields(self.view)
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/django_filters/rest_framework/backends.py", line 139, in get_schema_fields
    ) for field_name, field in filterset_class.base_filters.items()
  File "/Users/jesusch/git/jobq-api/venv/lib/python3.7/site-packages/django_filters/rest_framework/backends.py", line 139, in <listcomp>
    ) for field_name, field in filterset_class.base_filters.items()
KeyError: 'required'

[jesusch]

This is my FilterSetClass:

import django_mongoengine_filter
from . import models
class JobFilter(django_mongoengine_filter.FilterSet):

    jobtype = django_mongoengine_filter.StringFilter()
    clientid = django_mongoengine_filter.NumberFilter()
    jobstatus = django_mongoengine_filter.StringFilter()

    class Meta:
        model = models.Job
        fields = ['jobtype', 'clientid', 'jobstatus']

my ViewSet:

from rest_framework_mongoengine import viewsets
from . import models, serializers, filters

class JobViewSet(viewsets.ReadOnlyModelViewSet):
    lookup_field = 'id'
    queryset = models.Job.objects()
    serializer_class = serializers.JobSerializer
    filterset_class = filters.JobFilter

My Serializer:

from rest_framework_mongoengine.serializers import DocumentSerializer, DynamicDocumentSerializer
from . import models

class JobSerializer(DynamicDocumentSerializer):
    """This is Some fancy blaaaa"""

    class Meta:
        model = models.Job
        fields = '__all__'

And finally the Model:

from datetime import datetime
from mongoengine import fields
from django_mongoengine.document import DynamicDocument

class Job(DynamicDocument):

    meta = {'db_alias': 'default', 'collection': 'jobqueue', 'strict': False}

    jobtype = fields.StringField(required=True)
    jobstatus = fields.StringField(required=True)
    jobstartat = fields.DateTimeField(default=datetime.now())
    clientid = fields.IntField()

[barseghyanartur]

Thanks! I'll take a look.

[jesusch]

If you need any assistance let me know. I'm eager to help. It somehow looks that django_filters/rest_framework/backends tries to resolve the fields against it's own Filters Class - and not using django-mongoengine-filter.Filters

[jesusch]

ah - no it's really this call in that backend:

compat.coreapi.Field(
                name=field_name,
                required=field.extra['required'],
                location='query',
                schema=self.get_coreschema_field(field)

it is relying on that required being in self.extra - not self.required

[barseghyanartur]

No problem. Will be fixed.

        self.extra = kwargs
        self.extra.setdefault('required', False)

I'll do this.

[jesusch]

yeah - but be aware, that self.required is used at other places e.g.: https://github.com/jesusch/django-mongoengine-filter/blob/master/django_mongoengine_filter/filters.py#L97

[jesusch]

I've done a PR which solves this issue locally

[jesusch]

ah - sorry.. in my branch I did the changes on master branch oO