Creating elastic search index failuring

[dagi-yadeta]

I followed saleor deployment on heroku described on url: https://docs.getsaleor.com/docs/deployment/heroku/

I followed all the steps under each subtopic outlined in the whole schema of instructions (configuration, deployment, preparing the database, ect). But all but the last step failed to succeed, i.e creating elasticsearch search index. I got the attached error while executing the command. May I please have some help?

Thanks in advance! search_index_failure.txt

[khashashin]

I have now the same issue, this command heroku run python manage.py search_index --create throws following error:

Creating index 'orders'
WARNING elasticsearch PUT https://ivy-75841735.eu-west-1.bonsaisearch.net:443/orders [status:400 request:0.042s] [PID:4:MainThread]
Traceback (most recent call last):
  File "manage.py", line 10, in <module>
    execute_from_command_line(sys.argv)
  File "/app/.heroku/python/lib/python3.7/site-packages/django/core/management/__init__.py", line 381, in execute_from_command_line
    utility.execute()
  File "/app/.heroku/python/lib/python3.7/site-packages/django/core/management/__init__.py", line 375, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/app/.heroku/python/lib/python3.7/site-packages/django/core/management/base.py", line 323, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/app/.heroku/python/lib/python3.7/site-packages/django/core/management/base.py", line 364, in execute
    output = self.handle(*args, **options)
  File "/app/.heroku/python/lib/python3.7/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 128, in handle
    self._create(models, options)
  File "/app/.heroku/python/lib/python3.7/site-packages/django_elasticsearch_dsl/management/commands/search_index.py", line 84, in _create
    index.create()
  File "/app/.heroku/python/lib/python3.7/site-packages/elasticsearch_dsl/index.py", line 203, in create
    self.connection.indices.create(index=self._name, body=self.to_dict(), **kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/elasticsearch/client/utils.py", line 76, in _wrapped
    return func(*args, params=params, **kwargs)
  File "/app/.heroku/python/lib/python3.7/site-packages/elasticsearch/client/indices.py", line 88, in create
    params=params, body=body)
  File "/app/.heroku/python/lib/python3.7/site-packages/elasticsearch/transport.py", line 318, in perform_request
    status, headers_response, data = connection.perform_request(method, url, params, body, headers=headers, ignore=ignore, timeout=timeout)
  File "/app/.heroku/python/lib/python3.7/site-packages/elasticsearch/connection/http_urllib3.py", line 186, in perform_request
    self._raise_error(response.status, raw_data)
  File "/app/.heroku/python/lib/python3.7/site-packages/elasticsearch/connection/base.py", line 125, in _raise_error
    raise HTTP_EXCEPTIONS.get(status_code, TransportError)(status_code, error_message, additional_info)
elasticsearch.exceptions.RequestError: RequestError(400, 'mapper_parsing_exception', 'Root mapping definition has unsupported parameters:  [doc : {properties={user_email={type=text}, discount_name={type=text}, last_name={type=text}, user={analyzer=email_analyzer, type=text}, first_name={type=text}}}]')
(env)

[karolkielecki]

Hey @dagi-yadeta , @khashashin ! Thank you for the report. It seems it's a bug. We'll take a look into this and see if we can find the cause. We'll try to do that as soon as possible but can't tell you when exactly. Thanks for your patience.

[dagi-yadeta]

@karolkielecki, @maarcingebala thank you for taking up the issue. Looking forward to the solutions!

[krzysztofwolski]

We are dropping integrated elasticsearch support right now since we use Postgres for all of our queries. - mirumee/saleor#5043 Even if you are looking for full text search you can achieve it using Postgres : https://www.postgresql.org/docs/9.5/textsearch.html

@dagi-yadeta if theres any particular usage of ES you are looking for let us know

[dagi-yadeta]

@krzysztofwolski thank you for considering my question. I used to use elasticsearch before that was why I tried add elasticsearch plugin during deploying saleor on heroku. If Postgres provides enhanced search there won't be any need to go for elasticsearch. I also managed to have saleor up and running on heroku leaving out elasticsearch :)

Sorry for being out of topic but I was trying to deploy saleor and dashboard2 for saleor on heroku but it is being rejected by heroku :( I have tried different ways to work around with no luck :( I have tried both from my local machine using the deployment instructions for heroku and using single button push for deploying to heroku, hyperlinked to your site. I don't know what I am missing. Is there any update to your documentation please. I can see dashboard 2 is built in typescript not python.

Create app App creation failed. Please ensure you have valid values in the above form fields.

/tmp/build_2ffe984b76745b1bf2c7ff2fc496cf5c/requirements.txt (line 34)) In --require-hashes mode, all requirements must have their versions pinned with ==. These do not: setuptools>=39.2.0 from https://files.pythonhosted.org/packages/54/28/c45d8b54c1339f9644b87663945e54a8503cfef59cf0f65b3ff5dd17cf64/setuptools-42.0.2-py2.py3-none-any.whl#sha256=c8abd0f3574bc23afd2f6fd2c415ba7d9e097c8a99b845473b0d957ba1e2dac6 (from cairocffi==1.1.0->-r /tmp/build_2ffe984b76745b1bf2c7ff2fc496cf5c/requirements.txt (line 34)) ! Push rejected, failed to compile Python app. ! Push failed

[dagi-yadeta]

@karolkielecki @krzysztofwolski @krzysztofwolski fortunately I managed to resolve the issues I was facing regarding deployment on heroku. The errors were related to the 'requirements_dev.txt' and 'requirements.txt' file. In the files all add-ons include hashes and for some reason heroku was blocking the saleor application when I pushed. After generating these text files with the below poetry commands, without hashes, the deployment went through successfully.

poetry export -f requirements.txt -o requirements_dev.txt --dev --without-hashes poetry export -f requirements.txt -o requirements.txt --without-hashes

I hope it will be of some use to someone out there facing similar issues :)

Cheers!

[grll]

since support for elasticsearch has been dropped maybe the section of the doc about it should be dropped as well: https://docs.saleor.io/docs/integrations/elasticsearch/

[macrekpy]

I was working on the extension of Saleor's support for elasticsearch for v.>=7, because Heroku deprecated elasticsearch v.<7 in their free Sandbox.

To extend compatibility you have to:

• follow new mapping requiremets for elasticsearch v.7.0 and higher, i.e. https://www.elastic.co/guide/en/elasticsearch/reference/current/mapping.html, https://django-elasticsearch-dsl.readthedocs.io/en/latest/es_index.html.

• upgrade elasticsearch, elasticsearch-dsl and django-elasticsearch-dsl to most recent versions (as of 05/22/2020)

• edit saleor/search/documents.py

Old saleor/search/documents.py should be changed accordingly:

from django_elasticsearch_dsl import Document, Index, fields
from elasticsearch_dsl import analyzer, token_filter
from django_elasticsearch_dsl.registries import registry

from ..account.models import User
from ..account.utils import get_user_first_name, get_user_last_name
from ..order.models import Order
from ..product.models import Product

storefront = Index('storefront')
storefront.settings(number_of_shards=1, number_of_replicas=0)

partial_words = token_filter(
    'partial_words', 'edge_ngram', min_gram=3, max_gram=15)
title_analyzer = analyzer(
    'title_analyzer',
    tokenizer='standard',
    filter=[partial_words, 'lowercase'])
email_analyzer = analyzer('email_analyzer', tokenizer='uax_url_email')

@storefront.document
class ProductDocument(Document):
    class Django:

        model = Product
        fields = [
            'name', 
            'description', 
            'is_published',
            ]

Following indices (i.e. users and orders) have to be optimized as well.

It works fine with Heroku bonsai-elasticsearch 7.2.0.

[dagi-yadeta]

@macrekpy thank you! I will try out the solution.