barseghyanartur / django-elasticsearch-dsl-drf

Integrate Elasticsearch DSL with Django REST framework.
https://pypi.python.org/pypi/django-elasticsearch-dsl-drf
368 stars 117 forks source link

_type parameter is deprecated in the MoreLikeThis query #269

Open Prakash2403 opened 3 years ago

Prakash2403 commented 3 years ago

Describe the bug _type parameter is deprecated in MoreLikeThis query in the latest version of ES. I get the following deprecation warning after I query at more_like_this endpoint

{
  "type": "deprecation",
  "timestamp": "2021-08-22T18:00:17,324Z",
  "level": "DEPRECATION",
  "component": "o.e.d.i.q.MoreLikeThisQueryBuilder",
  "cluster.name": "docker-cluster",
  "node.name": "2fd6f0e3e02f",
  "message": "[types removal] Types are deprecated in [more_like_this] queries. The type should no longer be specified in the [like] and [unlike] sections.",
  "cluster.uuid": "iDcaco5NROutaFg3svGMrw",
  "node.id": "hdeF_22aSxWUswmsMhYC9A"
}

Environment

Python environment:

  1. pip list
    Package                       Version
    ----------------------------- ---------
    amqp                          2.6.1
    asgiref                       3.4.1
    attrs                         21.2.0
    billiard                      3.6.4.0
    boto3                         1.17.93
    botocore                      1.20.112
    celery                        4.4.6
    certifi                       2021.5.30
    cffi                          1.14.6
    chardet                       4.0.0
    click                         8.0.1
    click-plugins                 1.1.1
    cligj                         0.7.2
    coreapi                       2.3.3
    coreschema                    0.0.4
    cryptography                  3.4.7
    defusedxml                    0.7.1
    diff-match-patch              20200713
    Django                        3.2.4
    django-celery-results         2.0.1
    django-cors-headers           3.7.0
    django-elasticsearch-dsl      7.2.0
    django-elasticsearch-dsl-drf  0.22.1
    django-import-export          2.5.0
    django-nine                   0.2.4
    django-templated-mail         1.1.1
    djangorestframework           3.12.4
    djangorestframework-simplejwt 4.8.0
    djoser                        2.1.0
    elasticsearch                 7.14.0
    elasticsearch-dsl             7.4.0
    et-xmlfile                    1.1.0
    Fiona                         1.8.20
    flower                        0.9.7
    future                        0.18.2
    geopandas                     0.9.0
    gunicorn                      20.1.0
    humanize                      3.11.0
    idna                          2.10
    itypes                        1.2.0
    Jinja2                        3.0.1
    jmespath                      0.10.0
    kombu                         4.6.11
    lxml                          4.6.3
    MarkupPy                      1.14
    MarkupSafe                    2.0.1
    munch                         2.5.0
    numpy                         1.21.2
    oauthlib                      3.1.1
    odfpy                         1.4.1
    pandas                        1.3.2
    pathvalidate                  2.4.1
    Pillow                        8.2.0
    pip                           21.2.4
    prometheus-client             0.8.0
    psycopg2-binary               2.8.6
    pycparser                     2.20
    PyJWT                         2.1.0
    pyproj                        3.1.0
    pytesseract                   0.3.7
    python-dateutil               2.8.2
    python3-openid                3.2.0
    pytz                          2021.1
    PyYAML                        5.4.1
    redis                         3.5.3
    requests                      2.25.1
    requests-oauthlib             1.3.0
    s3transfer                    0.4.2
    setuptools                    57.4.0
    Shapely                       1.7.1
    six                           1.16.0
    social-auth-app-django        4.0.0
    social-auth-core              4.1.0
    sqlparse                      0.4.1
    tablib                        3.0.0
    tornado                       6.1
    unicodecsv                    0.14.1
    uritemplate                   3.0.1
    urllib3                       1.26.6
    vine                          1.3.0
    wheel                         0.37.0
    xlwt                          1.3.0
  2. python --version 3.9.6

Which version of Elasticsearch are you using? 7.12.1

To Reproduce Steps to reproduce the behavior:

  1. Create a ViewSet and inherit MoreLikeThisMixin
  2. Run a query against more_like_this endpoint
  3. Monitor ES server logs

Expected behavior Deprecation warnings shouldn't appear

@barseghyanartur I've fixed it on my fork. If you feel that this should be merged here, kindly let me know. I'll make a PR

citizenfish commented 1 year ago

Leaving this note here for anyone trying to get more_like_this working with Elasticsearch 7 or higher. I could not get it to work in the AWS version of Elasticsearch. The fix proposed by @Prakash2403 works, my solution was to use a local version of the class:-

from rest_framework.decorators import action
from elasticsearch_dsl.query import MoreLikeThis
from rest_framework.response import Response

import copy

"""
The current version of django-elasticsearch-dsl-drf will not work with Elasticsearch 7 or higher more_like_this
reason being that the _type parameter is deprecated. So we copy their code and fix it locally

"""

class MoreLikeThisMixin(object):
    """More-like-this mixin."""

    @action(detail=True)
    def more_like_this(self, request, pk=None, id=None):
        """More-like-this functionality detail view.

        :param request:
        :return:
        """
        if 'view' in request.parser_context:
            view = request.parser_context['view']
            kwargs = copy.copy(getattr(view, 'more_like_this_options', {}))
            id_ = pk if pk else id

            queryset = self.filter_queryset(self.get_queryset())
            fields = kwargs.pop('fields', [])

            if fields:
                queryset = queryset.query(
                    MoreLikeThis(
                        fields=fields,
                        like={
                            '_id': "{}".format(id_),
                            '_index': "{}".format(self.index)
                        },
                        **kwargs
                    )
                ).sort('_score')
            else:
                queryset = queryset.query(
                    MoreLikeThis(
                        like={
                            '_id': "{}".format(id_),
                            '_index': "{}".format(self.index)
                        },
                        **kwargs
                    )
                ).sort('_score')

            # Standard list-view implementation
            page = self.paginate_queryset(queryset)
            if page is not None:
                serializer = self.get_serializer(page, many=True)
                return self.get_paginated_response(serializer.data)

            serializer = self.get_serializer(queryset, many=True)
            return Response(serializer.data)