GraphQL Search Request - Internal Error on Magento 2.3.4

[fjmiguel]

By developing using GraphQL, Elastiscsuite tries to retrieve the index name from the Search Request (graphql_product_search). Which is not declared on vendor/smile/module-elasticsuite-catalog/etc/elasticsuite_search_request.xml. Therefore, this is not mapped on Elasticsearch as well.

Preconditions

Magento Version : EE 2.3.4 ElasticSuite Version : 2.8.4 Environment : Developer and Production Third party modules : -

Steps to reproduce

1. Create a /graphql post/request (using Postman for example).
2. Check the error response.
3. Check Magento logs.

Expected result

GraphQL request should work as expected.

Actual result

The response is an Internal Error using Postman for example.

Magento exception log throws:

[2020-02-04 16:06:27] report.ERROR: index does not exist yet. Make sure everything is reindexed. {"exception":"[object] (GraphQL\\Error\\Error(code: 0): index does not exist yet. Make sure everything is reindexed. at /var/www/html/src/vendor/webonyx/graphql-php/src/Error/Error.php:174, LogicException(code: 0): index does not exist yet. Make sure everything is reindexed. at /var/www/html/src/vendor/smile/elasticsuite/src/module-elasticsuite-core/Index/IndexOperation.php:117)"} []

Thanks,

[idziakjakub]

Hi @fjmiguel, I have the same problem with Magento 2.3.4 version.

[romainruaud]

Hi guys, we'll prioritize this issue since we don't want to break GraphQL compatibility.

If you have any workaround in the meantime, do not hesitate to propose it here. And of course, if you have a fix, any PR will be welcome.

regards

[mercs600]

Faced the same issue

[fjmiguel]

@romainruaud just to speed up things: we added here on src/vendor/smile/elasticsuite/src/module-elasticsuite-catalog/etc/elasticsuite_search_request.xml `

Smile\ElasticsuiteCatalog\Model\Product\Search\Request\Container\Filter\Stock Smile\ElasticsuiteCatalog\Model\Product\Search\Request\Container\Filter\VisibleInSearch Smile\ElasticsuiteCatalog\Model\Product\Search\Request\Container\Filter\SearchBlacklist

    <aggregations>
        <provider name="filterableAttributesProvider">searchFilterableAttributesProvider</provider>
        <aggregation xsi:type="termBucket" name="attribute_set_id" field="attribute_set_id"/>
    </aggregations>
</request>`

And on file src/vendor/smile/elasticsuite/src/module-elasticsuite-catalog/etc/elasticsuite_indices.xml inside node index identifier="catalog_product" defaultSearchType="product": <field name="price.from" type="double" nestedPath="price" /> <field name="price.to" type="double" nestedPath="price" />

So far looks good. We are still testing it. But maybe this can be helpful for you.

[fjmiguel]

Just a short update regarding the above: Even after map some fields, we are still facing some issues when we start to play around with GraphQL. If we have some news or workaround we will let you know as well.

[romainruaud]

Yes, there is more work to be done, I expect to deliver it asap.

In any cases, considering the current state of GraphQL implementation in Magento backend, I am not being optimistic on building something on top of it when having a high volume catalog.

Just take a look at all the mess in this implementation : https://github.com/magento/magento2/blob/2.4-develop/app/code/Magento/CatalogGraphQl/Model/Resolver/Products/Query/Search.php#L85

"Current page must be set to 0 and page size to max for search to grab all ID's as temporary workaround" => This works well with the Venia sample data considering they have only hundreds of products, but shall you have 100K+ products, you'll face huge performance issues, if not a fatal error due to memory limit reached.

That being said, we must stay compatible with Magento's implementation, even if their resolver looks risky for now.

About price filtering, @fjmiguel , I'm not sure it's working at all on a legacy Magento implementation (look on https://venia.magento.com/ : all the filters seems to have nearly no effect).

If you are building something which is completely custom, I'd like to have a look on your GraphQL query.

Regards

[romainruaud]

@fjmiguel and @idziakjakub can you try the fixes provided in this PR : https://github.com/Smile-SA/elasticsuite/pull/1699

It should be working with Magento 2.3.4 and also prevent him to query the engine with their "max page size" hacky thing.

This has also been tested against 2.3.3 : it will fallback to legacy Resolver.

Due to the fact their GraphQL implementation has completely changed in 2.3.4, I was forced to introduce this "fallback to legacy" behavior for older versions.

Let me know if you encounter issues.

[idziakjakub]

@romainruaud

I just tested GraphQL queries and there is a problem with filter product by URL key - it's used in PWA Studio to load one product data on product page.

Example query:

{
    productDetail: products(filter: { url_key: { eq: "product-url-key" } }) {
        items {
            __typename
            id
            name
            sku
            url_key
        }
    }
}

I see in file https://github.com/Smile-SA/elasticsuite/pull/1699/files#diff-ac477b81780d3461a3ed5a577c2c1d93 there is hardcoded filter by category ID.

In Magento 2.3.4 GraphQL schema there are more attributes used to filter products (Brand is our custom attribute):

There is more info about it in Magento documentation: https://devdocs.magento.com/guides/v2.3/graphql/queries/products.html#ProductFilterInput

[romainruaud]

@idziakjakub there is no hardcoded filter on category_id, I'm just testing if it's present to update the current Elasticsuite context. If there is no filter on category_id, this code should not been triggered.

What is exactly the issue you are facing with the filter product by URL key ?

Regards

[idziakjakub]

@romainruaud I'm getting an empty response instead of requested product data:

{
  "data": {
    "productDetail": {
      "items": []
    }
  }
}

Requests with filter by SKU or price range are working fine.

[romainruaud]

Well it might be due to the fact that url_key attribute is not searchable => so it's not indexed into Elasticsearch.

Can you try to set it as "used_in_search=1" in the database and process a full reindex ?

If that was the problem, we should enforce indexing this attribute in later releases if using GraphQL.

regards

[idziakjakub]

I think url_key is set to searchable in Magneto by default - in my database, it is set as "used_in_search=1".

There is example product from ElasticSearch:

url_key is added to "indexed_attributes".

Best regards

[romainruaud]

Okay, I'll have a deeper look on this one then.

Thank you for reporting.

I understood that the first batch of fixes is working either ?

[idziakjakub]

Yes, generally it fixes most of the problems with products/category queries.

Best regards

[jaiminmagento]

I am facing the same issue in Magento EE 2.3.4 and tried solution given but it gives another problem. each products search query gives same error. for e.g.

{
  products(search: "Yoga pants", pageSize: 10) {
    total_count
    items {
      name
      sku
      price_range {
        minimum_price {
          regular_price {
            value
            currency
          }
        }
      }
    }
    page_info {
      page_size
      current_page
    }
  }
}

gives below error response

{
  "errors": [
    {
      "debugMessage": "The \"search_term\" attribute name is invalid. Reset the name and try again.",
      "message": "Internal server error",
      "extensions": {
        "category": "internal"
      },
      "locations": [
        {
          "line": 2,
          "column": 3
        }
      ],
      "path": [
        "products"
      ]
    }
  ],
  "data": {
    "products": null
  }
}

Can you please provide the solution ASAP.

[romainruaud]

Can you please provide the solution ASAP.

No. @jaiminmagento this is an Open Source projet here. So please don't be rude and don't ask for providing solutions ASAP since you are already taking benefits or everything published here for free.

That being said, I acknowledge there are still issues on my PR branch for this issue, I'll have a look on them.

Regards,

[jaiminmagento]

Hi @romainruaud, I understand this but our live site is broken as we are using AEM as frontend and Magento as backend and everything is based on Graphql. I hope you understand and my apology my intention is not being rude just if you can give priority we can solve our broken site issue.

[romainruaud]

Hi @jaiminmagento,

our Open Source support policy (here on Github) does not cover any priority mechanism for issues opened by users, for the reason that everything here is provided for free by Smile SA.

If you (or your client) is using Elasticsuite on a production website and is in need for prioritization of bugfixes by our side, you might be more comfortable purchasing a commercial (paid) support contract with us. That's something we can provide to you as part of our Commercial support policy, and that can guarantee your a proper escalation process and also a reduced response time from us. If you are using an intensive usage of Elasticsuite for Enterprise Grade websites (I guess it's the case since you are using AEM as frontend), that's probably something you may want to consider.

Let's continue this discussion by mail if you feel the need for, by writing to me directly at romain.ruaud@smile.fr

Regards

[romainruaud]

That being said, back to the issues :

• I've adressed the search_term issue on my last commit on the PR branch, feel free to test it.

I'm gonna have a look on the url_key issue now.

regards

[romainruaud]

@idziakjakub my PR is updated with a fix for url_key : the field has to be declared as filterable so that building the ES query filter on it will succeed.

Can you check this (you have to clean all cache & reindex before) ? This should be enough.

In any cases, I think I'll rework everything related to GraphQL on next major Elasticsuite release (2.9.0) and might move it as a separated module rather than being in ElasticsuiteCatalog. I'll probably have to also check that any potential attribute being used for filtering though graphQL is properly set as being filterable for Elasticsearch : previously, only attributes configured with "Filterable in search" for the layered navigation were.

Let me know how is these fixes going.

Regards

[idziakjakub]

@romainruaud - I just tested get a product by url_key and it's working fine, thanks!

If you need to test all types of queries there are examples for sample data in Magento documentation:

https://devdocs.magento.com/guides/v2.3/graphql/queries/products.html

Best regards Jakub

[romainruaud]

Okay,

I also refactored these parts :

• "aggregations" output are now properly fetched from the Elasticsearch query

• "LayerFilter and LayerFilterItem" output are also properly fetched from the Elasticsearch query (and omg what a mess they are on a legacy Magento...)

This is also much faster now since it relies on the Elasticsearch query to be built.

Due to Backward Compatibility completely broken between 2.3.4 and older versions, the code actually contains some hacks to be compatible with any 2.3.x version.

We'll probably be more strict on next release and allow only Magento >= 2.3.4 so please be advise that you should remain up-to-date if you plan to build things on Magento GraphQL + ElasticSuite.

[romainruaud]

Fixed by #1699 and will be part of next release.

[rbayet]

Due to Backward Compatibility completely broken between 2.3.4 and older versions, the code actually contains some hacks to be compatible with any 2.3.x version.

We'll probably be more strict on next release and allow only Magento >= 2.3.4 so please be advise that you should remain up-to-date if you plan to build things on Magento GraphQL + ElasticSuite.

Hi anyone wandering here,

Please be aware of the warning we've added to our 2.8.5 release note. Unfortunately, we introduced a >= Magento 2.3.2 requirement by relying on the native class \Magento\CatalogSearch\Model\ResourceModel\Fulltext\Collection\SearchResultApplier which does not exist in 2.3.0 and 2.3.1. The error you should encounter is Source class "\elasticsuiteSearchResultApplier" for "elasticsuiteSearchResultApplierFactory" generation does not exist..

For the time being, if you don't rely at all on GraphQL + Elasticsuite, you are invited to stick to Elasticsuite 2.8.4 for the time being. If you do, we invite you to update at least up to Magento 2.3.2.

Regards,

[kamzata]

CatalogSearch\Model\ResourceModel\Fulltext\Collection\SearchResultApplier

I'm using Magento 2.3.4, ElasticSearch 6.8.7, and ElasticSuite 2.8.5 but I'm still facing this issue:

var/log/debug.log main.DEBUG: Source class "\elasticsuiteSearchResultApplier" for "elasticsuiteSearchResultApplierFactory" generation does not exist. {"exception":"[object] (RuntimeException(code: 0): Source class \"\\elasticsuiteSearchResultApplier\" for \"elasticsuiteSearchResultApplierFactory\" generation does not exist. at /var/www/magento/vendor/magento/framework/Code/Generator.php:222)"} []

By the way, this error message is shown even if ENABLE DEBUG MODE in Magento Backend is set to NO