All Products Disappear After Update of single Product

duffner commented 3 years ago

We are noticing an issue when reassigning products, from one category to another. When we move a product from one category to another, the indexes are invalidated, but for some reason, all products are deleted from the website until the index refreshes.

https://github.com/Smile-SA/elasticsuite/issues/1075

The same Issue Experienced. ^^^

Preconditions

Magento Multi-Website, and Multiple Root Level Catalogs 100k products Most Products Are configurable, not sure if this relates or not. Only Configurable Products are Assigned to a Category, Simples are not. I'm including this, because I'm curious if something with ElasticIndexing states that a product is not accessible until all children are available. But Might be completely wrong.

Magento Version : 2.4.1

ElasticSuite Version : 2.10.3

Environment : Production

Third party modules :

Steps to reproduce

Edit product SKU 58432.
Product was assigned to “All Brands -> Departments” and “All Brands -> Departments -> Equine + Livestock”. I unchecked these two categories.
Product also remains assigned to “All Brands -> Storage Solutions -> Hay Storage”, “DealerPortal -> Departments”, “DealerPortal -> Departments -> Equine + Livestock” – those were not changed
I assigned the product to “All Brands -> Departments -> Equine + Livestock -> Run-In Sheds”.
Save.
Repeat same with SKU 58542.
Indexes invalidated. I watched them process and all go back to a Ready status in the Admin panel.

Expected result

After reindex completed, products should have been removed from their old categories and assigned to the new.
No other items on the site are expected to be impacted.

Actual result

Completed products are missing from categories until a full reindex is performed

The exact steps include actual SKUs in our Database, but these are more for our internal reference of this issue.

reviskar commented 3 years ago

Hi! Can confirm, exactly the same problem occurring here, with identical flow and exactly the same versions. @duffner any chance you had any luck figuring this out? I've also narrowed it down being related with Elasticsuite, but having a hard time pinpointing the exact failure mechanism due to it being a production issue for us mainly. We changed all indexers back to "scheduled" and that somewhat helped drop the random occurrancy-rate of the problem, but it still does occur.

duffner commented 3 years ago

@reviskar Unfortunately no, we still have the issue from time to time. Are you self-hosting Elastic, or using a service such as Amazon Elasticsearch Service?

reviskar commented 3 years ago

The local web hosting service we use is responsible for hosting the elasticsearch service, but take it as a self-hosting Elastic as it is installed on the same private server.

romainruaud commented 3 years ago

@vahonc can you try to reproduce this one after your current tasks ?

vahonc commented 3 years ago

Ok, I'll try to investigate this issue.

ia-gaurav commented 3 years ago

hi @vahonc seems issue for me also i am using Magento 2.4.1 with most configurable products, most of times my products disappers i have to do reindexing to show then again on listing page

vahonc commented 3 years ago

Hello all,

I've tried to reproduce this issue on two local environments with the latest version of Magento EE (2.4.1, 2.4.2) and ES 2.10.3. In both cases, it was a clean version of Magento with default sample data and without any third-party modules.

So I did some tests with a few configurable products (like SKU: MT12) and tried to assigned/unassigned/add to different categories and I didn't notice any issues with missing these products. It was successfully removed from all old categories and displayed in the new category. I'm not sure but it could depend on some specific structure of the catalog tree (start root point, multi-levels, etc.) or some indexes (price, products indexes) based on which ES builds their indexes, it also somehow could be linked with Cron.

Anyway, we need more information from you guys, about your custom or specific configurations to be able more accurately reproduce or understand this issue.

@duffner, could you please more detailed describe your configuration and the result which you have now? Are the products missing from all categories witch they were assigned? Check the product types and their configurations (visibility), root-level configuration. If it's possible try to check this issue on some test products and categories.

@ia-gaurav please also add more details about your test conditions. Are the products missing after re-assigned from one category to another, or due to some other processes?

BR, Vadym

ia-gaurav commented 3 years ago

hi @vahonc - i was monitoring same for last 2 day , i did not got issue .

Earlier i guess issue was due to - i was using same Indices Alias Name on 2 website on same server.

I am still monitoring i did not faced issue from last 2 days.

vahonc commented 3 years ago

@ia-gaurav, thanks for your feedback!

BR, Vadym

reviskar commented 3 years ago

@vahonc thanks for investigating. Did you try it in a multi-store environment? For us it still happens regularly (about once a week, because of our custom product importer cron), but it was fairly reproducible when elasticsuite indexes were set to "On Save" behavior:

Start the indexer
Binge-save a product category relation in the admin.
Eventually would result in products dissapearing, usually accompanied with a indexer deadlock log.

[2021-02-26 12:20:23] main.ERROR: Cron Job indexer_reindex_all_invalid has an error: SQLSTATE[40001]: Serialization failure: 1213 Deadlock found when trying to get lock; try restarting transaction, query was: INSERT INTO catalog_category_product_index_store4_tmp ...

@duffner any chance you have any Amasty modules installed? Maybe one or two of them could also be an issue. I'll try ruling them in/out aswell once I have time.

vahonc commented 3 years ago

Hello @reviskar,

If I'm not wrong when you use a Cron for a custom product importer you should change the mode of all your indexes (or most important like product, price, catalogsearch, etc) to Update by Schedule.

Also, I think that a mysql deadlock it smth definitely out of the ES module but I agree with you that some third-party modules like Amasty could impact this issue.

BR, Vadym

reviskar commented 3 years ago

@vahonc that's what we did, but for reproducing purposes it allowed us to kind-of reproduce the issue. The problem still stands even with scheduling, that it seems like the indexer crash makes magento use another (newly created) index that is empty.

vahonc commented 3 years ago

@duffner, could you please more detailed describe your configuration and the result which you have now? Are the products missing from all categories witch they were assigned? Check the product types and their configurations (visibility), root-level configuration. If it's possible try to check this issue on some test products and categories.

@duffner, could you please provide us more details here, because we need some explanations for better understanding a start point for reproducing?

BR, Vadym

duffner commented 3 years ago

Hi @vahonc, the issue is that all products are missing from all categories.

All products are Configurable products that are set at the root level category and child categories, however, the varients are set to not visible individually and are also not assigned to any categories.

Is there anything else that would be helpful?

My theory is as follows:

Configurable product's category assignment is updated, indexers are invalidated
Products Category Assignment is removed
Categories are now empty, because the index was invalidated
Category Index is regenerated
All Categories are Empty because no products exist (yet)
Configurable index is run, which allows the configurable to display as they have active child products
Indexer for categories is now stuck broken

I feel that the issue relates to indexes being truncated before new ones are available to take their place. And I think the issue might be specific to configurable products when Child Products are not assigned to a category. As it appears the logic truncates the category assignment when it's replaced with the new one, but the new one doesn't have configurable product assignment.

Esteraa commented 3 years ago

I think we have the same issue on M2.4.2. Not sure what causes the issue. In our case we have simple products and we dont have any Amasty modules.

After the reindex the products are shown again in the backend and frontend.

vahonc commented 3 years ago

Hello @duffner,

I feel that the issue relates to indexes being truncated before new ones are available to take their place. And I think the issue might be specific to configurable products when Child Products are not assigned to a category. As it appears the logic truncates the category assignment when it's replaced with the new one, but the new one doesn't have configurable product assignment.

Yes, this is possible. Please check in the Magento logs (exception.log or system.log) to see if there are any logged errors during re-indexing.

BR, Vadym

duffner commented 3 years ago

I don’t see anything at the current time in the logs. Unfortunately, my issue is not consistently reproducible. I will however check the logs once it happens again.

ssx commented 3 years ago

There's a Magento patch which has just been updated which may be related:

https://support.magento.com/hc/en-us/articles/360050636011-MDVA-30977-Magento-patch-missing-products-from-categories-indexing-related

vahonc commented 3 years ago

@duffner, @Esteraa, @reviskar could you please check if the MDVA-30977 patch will solve your problems?

BR, Vadym

Esteraa commented 3 years ago

@vahonc The customers who had this issue were already on M2.4.2. As it says the fix from the patch should be in M2.4.2.

Esteraa commented 3 years ago

@vahonc We found that after updating an attribute the products disappear. We also have this issue on our demo store (m2.4.2) where we dont have any other external modules than Smile ElasticSuite. The products disappeared after adding/changing product attributes.

I found an Elastic error, perhaps it's related:

[2021-04-30 12:58:43] main.ERROR: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_default_catalog_product_20210426_083113","node":"k37BBqHxQ4yKWjXfgNHJMw","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400} [] []
[2021-04-30 12:58:43] main.ERROR: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_default_catalog_product_20210426_083113","node":"k37BBqHxQ4yKWjXfgNHJMw","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400} [] []
[2021-04-30 12:58:46] main.ERROR: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_default_catalog_product_20210426_083113","node":"k37BBqHxQ4yKWjXfgNHJMw","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400} [] []
[2021-04-30 12:58:46] main.ERROR: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_default_catalog_product_20210426_083113","node":"k37BBqHxQ4yKWjXfgNHJMw","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400} [] []
[2021-04-30 12:58:50] main.ERROR: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_default_catalog_product_20210426_083113","node":"k37BBqHxQ4yKWjXfgNHJMw","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400} [] []
[2021-04-30 12:58:50] main.ERROR: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"magento2_default_catalog_product_20210426_083113","node":"k37BBqHxQ4yKWjXfgNHJMw","reason":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead.","caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [produc_type] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}}},"status":400} [] []

romainruaud commented 3 years ago

What kind of update did you proceed for your attribute @Esteraa ?

Esteraa commented 3 years ago

@romainruaud Just creating and assigning products to new attributes, it wasn't clear what caused it as we noticed the products weren't shown later. Another customer told me he added an attribute to to show in the layered navigation and another also told me he adjusted settings at an attribute. So to me it's not very clear what caused it, seems if you change anything to an attribute it's already triggered.

I can try to test some different actions on our demo environment this week, to see if I can reproduce.

meirzaicube commented 3 years ago

@Esteraa have you tried the patch from @ssx ? https://support.magento.com/hc/en-us/articles/360050636011-MDVA-30977-Magento-patch-missing-products-from-categories-indexing-related

I don't know if my situation is the same as you, this is my issue #2137 , after applying the patch the bug is fixed.

Thanks @ssx for the information.

Esteraa commented 3 years ago

@meirzaicube We are already on m2.4.2 where the patch is included.

ssx commented 3 years ago

@Esteraa I'm seeing it on a 2.4.2 installation as well.

duffner commented 3 years ago

We just had this occur on another project, using the same versions. Somewhere, it appears that when an index is invalidated, it's deleted before a new one occurs, likely due to a cron overlap. My guess is that you truncate existing indexes before new ones are generated, and if something goes wrong with the server, new ones are never created, and the old ones are deleted, which leads to empty categories. The projects we have issues with have Huge Indexes from configurable products and tons of categories and customer groups.

duffner commented 3 years ago

Team, I'd be willing to bet it's with the function >proceedIndexInstall, where inside it looks for old indexes and deletes them, but there isn't anything that I can see which ensures there is a new index to replace it. It appears this was handled for a different scenario by converting a deleteIndex, to a "cleanIndex" approach in a different model, however, this code does not leverage this type of feature.


    {
        $indexIdentifier = ThesaurusIndex::INDEX_IDENTIER;
        $indexName       = $this->indexSettingsHelper->createIndexNameFromIdentifier($indexIdentifier, $storeId);
        $indexAlias      = $this->indexSettingsHelper->getIndexAliasFromIdentifier($indexIdentifier, $storeId);
        $indexSettings   = ['settings' => $this->getIndexSettings($synonyms, $expansions)];
        $this->client->createIndex($indexName, $indexSettings);
        $this->indexManager->proceedIndexInstall($indexName, $indexAlias);
        $this->cacheHelper->cleanIndexCache(ThesaurusIndex::INDEX_IDENTIER, $storeId);
    }

asrar7787 commented 3 years ago

not sure how far this was progressed but I was facing similar issue for one client By looking into @Esteraa log the issue could be related to the topic described here https://magento.stackexchange.com/a/322623/50597

duffner commented 2 years ago

We haven't had any reports of this in a while, either this means this was resolved unintentionally through routine Smile Updates "Great" or the customer just hasn't created us a new ticket in a while "Not Great".

profadminuniserviss commented 2 years ago

We have 2.4.2. and having the same issue.

Before we have 2.4.0 and the situation was critical. With 2.4.2 upgrade situation improved, but still appeared a few times per month.

We have Amasty modules, Firebear product import/export module which can affect successful reindex, but I hope this problem exists for many large stores.

In our case 89K products, 1000 categories, ~2000 attributes and 3 stores.

In my case we have a fast fix to again show products in categories: kill the current reindex process and execute bin/magento indexer:reset.

In ~10 minutes products again available after FPC cache clean.

Nuranto commented 2 years ago

We had a similar issue. All our categories were empty. I did not find any error logs, but I saw that the catalog_product index on ES was like 50Mo instead of 800Mo... Reindexing catalogsearch_fulltext solve the issue, and products were back in categories.

The only log I found is this, but I think it is more a consequence than a cause :

[2022-05-05 19:58:08] main.ERROR: Cron Job indexer_update_all_views has an error: {“error”:{“root_cause”:[{“type”:“index_not_found_exception”,“reason”:“no such index [magento2_website2_store_catalog_product_20220505_141222]“,”index_uuid”:“_na_“,”index”:“magento2_website2_store_catalog_product_20220505_141222”}],“type”:“index_not_found_exception”,“reason”:“no such index [magento2_website2_store_catalog_product_20220505_141222]“,”index_uuid”:“_na_“,”index”:“magento2_website2_store_catalog_product_20220505_141222”},“status”:404}. Statistics: {“sum”:0,“count”:1,“realmem”:0,“emalloc”:0,“realmem_start”:672661504,“emalloc_start”:319430704} [] []
[2022-05-05 19:58:08] main.CRITICAL: {“error”:{“root_cause”:[{“type”:“index_not_found_exception”,“reason”:“no such index [magento2_website2_store_catalog_product_20220505_141222]“,”index_uuid”:“_na_“,”index”:“magento2_website2_store_catalog_product_20220505_141222”}],“type”:“index_not_found_exception”,“reason”:“no such index [magento2_website2_store_catalog_product_20220505_141222]“,”index_uuid”:“_na_“,”index”:“magento2_website2_store_catalog_product_20220505_141222”},“status”:404} {“exception”:“[object] (Elasticsearch\\Common\\Exceptions\\Missing404Exception(code: 404): {\“error\“:{\“root_cause\“:[{\“type\“:\“index_not_found_exception\“,\“reason\“:\“no such index [magento2_website2_store_catalog_product_20220505_141222]\“,\“index_uuid\“:\“_na_\“,\“index\“:\“magento2_website2_store_catalog_product_20220505_141222\“}],\“type\“:\“index_not_found_exception\“,\“reason\“:\“no such index [magento2_website2_store_catalog_product_20220505_141222]\“,\“index_uuid\“:\“_na_\“,\“index\“:\“magento2_website2_store_catalog_product_20220505_141222\“},\“status\“:404} at /var/www/html/vendor/elasticsearch/elasticsearch/src/Elasticsearch/Connections/Connection.php:667)“} []

romainruaud commented 2 years ago

We've seen recently similar issues on Smile projects : it was due to a failure happening during the catalog_category_product reindexing (a deadlock). This leads to "catalog_category_product_index" containing partial (or empty) data.

But the next indexer, which is the catalogsearch_fulltext, will runs anyway and will then index nothing or nearly nothing related to products because they are not considered as belonging to categories or being visible.

Do you have some evidences of failures from other indexes as well ?

Nuranto commented 2 years ago

I did not see any logs about that in my case.

However, in your case, it is a Magento issue then. In case of failure during catalog_category_product, there should be a transaction rollback, shouldn't it ?

romainruaud commented 11 months ago

2 years old issue -> I close.

Feel free to open a new one if needed.

Regards

Smile-SA / elasticsuite