Elastic Suite slow comparing to commercial extensions

[tomekjordan]

Preconditions

Why Elastic Suite is so slow in autocomplete comparing to Elastic commercial extensions from Amasty, Wyomind nad Mirasvit? which is blazing fast when you just try to search. Even comparing to old mysql Instant Search from Itoris or Mageworx - its just slow. Customers expect "algolia" style instant results, not results after 1-2seconds....

Magento Version : 2.3

ElasticSuite Version : latest

Environment : clean Magneto with sample data

Third party modules : NO

Steps to reproduce

1. Just install Elastic Suite on vanilla Magento 2.3
2. Install sample date - or even yours
3. Try to search with autocomplete

Expected result

1.

Actual result

1. [Screenshot, logs]

[romainruaud]

Dear ElasticSuite user.

We are always pleased to hear constructive remarks about our product.

However, I'm not completely sure about the objectivity of this one. As you may have already noticed, we are providing this extension for free here.

Actually, ElasticSuite is embedded into more than 1200 Magento2 websites across the world, and we are very proud of it.

At Smile, we have it running on many websites, some with hundreds of thousands products, even some with millions of products, and performances remains very fast.

With a properly sized production platform, we benchmarked several times that autocompletion results are rendered <0.5s and search result page is rendered <2s.

On an architectural point of view, we did chose to embed Elasticsuite as a Magento module and a legacy Search Engine implementation (like Magento does for legacy Elasticsearch or Solr adapters), meaning that :

• we are fully respecting the core principles of Magento
• any customer can benefit our solution without breaking anything or having to subscribe to a third-party SaaS platform, or any additional third-party server to host an additional application
• any Magento2 developer will understand easily how things are done in ElasticSuite.

That being said, such an implementation comes with drawbacks, especially because we are following a legacy Search Engine implementation, we are due to suffer the way Magento is internally built. Eg. for the autocomplete, most of latency you are seeing is due to the time spent by the Magento call-stack to run when hitting the Frontend Controller, and so on... Being tied to a legacy implementation, there are these kind of time-consuming resources that are hard to suppress.

Anyway, I'll be quite happy to review a Pull Request from you to help us achieving better performances.

Best regards.

[gewaechshaus]

Hi @romainruaud, Hi @tomekjordan,

we are facing this issue also, the autocomplete is kind of slow and unresponsive if you compare it with the mirasvit or wyomind one. Maybe it would be an option, to implement a backend configuration to choose a external payload which doesn't bootstrap the complete system like any conventional controller.

Anyway, I like ES - many thanks for providing such a great piece of code.

http://search-elastic-ultimate.m2.mirasvit.com/ http://elasticsearch.wyomind.com/

Cheers Jan

[gewaechshaus]

@romainruaud, @tomekjordan

There must be a bottleneck somewhere in the controller. I did some quick tests in one of our clients installations:

Note: We are using http2, so this if faster as every test in the demostore

Please type in the keyword "barf" at the search box located in the header. If you take a look at the console inspectors xhr request, you will see the request is finished after ~1500ms. [HTTP/2.0 200 OK 1453ms]

If you open the related URI in another Tab, you get significantly better results like: Cached result in 31ms: https://www.haustierkost.de/search/ajax/suggest/?q=barf

[romainruaud]

For me the bottleneck is the price rendering done in PHP, and the fact that it's done via a Magento controller.

If we could :

• move this to a REST Api endpoint
• just return some JSON and let the frontend render them properly

It would be a significant improvement.

And of course, if we were able to completely bypass the Magento stack, it would also be waaaay faster of course.

[gewaechshaus]

@romainruaud - I just inspected the wyomind one, they are sending a POST request to a payloader located under http://elasticsearch.wyomind.com/elastic.php

Did somebody ever perform kind of a profiling to find the bottleneck?

[gewaechshaus]

And for sure, many thanks for your fast reaction! Thumbs up.

[gewaechshaus]

So in conclusion we should build an external loader bypassing m2 combined with frontend rendered json...

[romainruaud]

yes, but you cannot "bypass" M2 easily. At least, there is something you don't want to bypass, it's the ES query building made by Elasticsuite, because it will :

• add any synonyms if accurate
• append optimizers function scoring to the query
• do a pre-fetch termVectors call to ES to know if the query should be fuzzy or not

And this Query building is made.. in a low-level PHP code wrapped into the Magento SearchEngineInterface implementation, so ...

[gewaechshaus]

Hmmm... BrainFactory is working :)

[gewaechshaus]

We need to inspect one of the commercial extensions :)

[gewaechshaus]

@romainruaud

And this Query building is made.. in a low-level PHP code wrapped into the Magento SearchEngineInterface implementation, so ...

Are you refering this model and the request/response builders in the search method?

https://github.com/Smile-SA/elasticsuite/blob/afa1a40a792fbba570afb0ef982b3c684979b45c/src/module-elasticsuite-core/Model/Search.php#L24

[southerncomputer]

@romainruaud it might be wise to add caching of AJAX request and disregard "space" to allow faster response time due to people backspacing and forward spacing eating up json calls! ` var cache = {}; $(".ac-remote-caching").autocomplete({ minLength: 2, source: function(request, response) { var term = request.term; if (term in cache) { response(cache[term]); return; }

        $.getJSON("../../../app-assets/data/jquery-ui/search.php", request, function(data, status, xhr) {
            cache[term] = data;
            response(data);
        });
    }
});`

[gewaechshaus]

Hey @southerncomputer - that sounds like a good plan also. I much appreciate your commitment around M2 generaly, we already had some conversion in other issue related threads here on github.

[gulatipriyanka]

@romainruaud Smile Elastic Suite search suggestion taking too much time to populate results in dropdown and is very slow. Please suggest how can we improve the speed or please provide if any patch is available for this.

Any help will be appreciated.

[simonrl]

Hey,

if anyone could share their findings on this one, it would be really nice. Maybe someone has already profiled autocomplete requests? This would save me some work.

I'd also happily take any hints on which part of the code could be optimized, and I will look into it.

Thanks!

[robsch]

The question of @tomekjordan is not necessarily criticism. I'm also wondering, what do the others do different, that they have such a fast response. I find this very crucial. Nowadays this is something that I would expect from a search engine. Waiting for a second after each key hit is annoying - at least for me, when I enter something into google or amazon or whereever, I expect a instant response. And I think I'm not allone...

This is no criticism. It doesn't make sense to criticize open source projects. It probably doesn't matter you, but personally I will tend to use one of the faster search modules, instead of ElasticSuite. What is a pity.

[davidwindell]

Hi @romainruaud would you consider re-opening this issue or creating a new one especially focused on the performance of the autocomplete to put efforts together on profiling and see if the community can come up with some understanding and suggestions for improving the response times?

[kpitn]

I love elasticsuite module, but i agree, defaut autocomplete way is a bit slow actually.

I try a way to speed up autocomplete in a new module available here : https://github.com/Web200/magento-elasticsuite-autocomplete

The goal is to avoid product loading, and use elasticsearch data to display product. No modification is done on js or phtml.

More information in module readme file.

[davidwindell]

@kpitn this is awesome, thanks for sharing - will check your module out next week.

[paugnu]

Hi @kpitn, do you think you could propose a PR to this repo? I think you've solved one of the main «issues» with ElasticSuite.

[jg-development]

+1

[Seppmann]

@kpitn We use ElasticSuite Thesaurus..which works fine without https://github.com/Web200/magento-elasticsuite-autocomplete. When we enable extension Thesaurus seems to be ignored and only "regular" search results are returned. Is there a way to include Thesaurus? Your extension does a fine job!

[kpitn]

Hi @Seppmann have you try the last version. If you don't active improve search. The search criteria come from elasticsuite module.

[Seppmann]

Hi @kpitn - thanks for your suggestion! I tried now on a vanilla M2.4.3-p2 version - only with sample data. I installed Elasticsuite and your Web200 autocomplete improvement. I saved one random synonym thesaurus like "laptop,man,men" Without your Extension the search results for "laptop" include all men/man products (autocomplete and search results). With your extension enabled and "improved search" set to yes: No "men/man products" are returned, only regular search hits. So thesaurus is ignored when your extension is enabled. If you have a way to fix this issue please let me know! Thanks and best regards!

[cptX]

I was playing a lot with autocomplete mechanism the last weeks. I can say that elasticsuite especially after the corrections @romainruaud did for the Greek Language is now one of the best out there (at least for my language). Today tried out the Mageplaza_Search module which comes with the Porto theme and although it is blazing fast it has 2 important issues:

1. It doesn't search without accents for the greek language (so it's useless)
2. It doesn't read the description of every product so you cannot search by a word that is in the description. (please correct me if I'm wrong)

So my only complain at the moment for elasticsuite is the delay there is at the first letters of typing. Initially I thought @romainruaud implemented it on purpose to have this delay to allow the user to type something before results come out to save unnecessary json calls. But after reading this thread here I now believe he did the best he could with the existing strategy. So if we want to eliminate this delay is it possible? @romainruaud ? Can we make it respond from the first characters and with less delay? This gives a really professional and quality feedback to the customer

I can provide the code for Mageplaza_Search for further study if anybody wants.