Duplicate multiselect values result in Product EAV indexing errors

[tdgroot]

Preconditions

1. Magento 2.2.2

Steps to reproduce

1. Create a multiselect attribute, fill it with values
2. Create an import sheet with a column for the multiselect attribute
3. Fill in duplicate values, for example Dutch|English|French|Chinese|English.
4. Import the sheet.

Expected result

$ bin/magento indexer:reindex
Design Config Grid index has been rebuilt successfully in 00:00:00
Customer Grid index has been rebuilt successfully in 00:00:00
Product Flat Data index has been rebuilt successfully in 00:00:00
Category Flat Data index has been rebuilt successfully in 00:00:00
Category Products index has been rebuilt successfully in 00:00:00
Product Categories index has been rebuilt successfully in 00:00:00
Product Price index has been rebuilt successfully in 00:00:00
Product EAV index has been rebuilt successfully in 00:00:00
Catalog Search index has been rebuilt successfully in 00:00:02
Stock index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:00
Catalog Product Rule index has been rebuilt successfully in 00:00:00
Amasty Customer Group Catalog Rule index has been rebuilt successfully in 00:00:00
Amasty Customer Group Catalog Rule index has been rebuilt successfully in 00:00:00
Amasty Customer Group Catalog Rule index has been rebuilt successfully in 00:00:00
Search Spell-Correction index has been rebuilt successfully in 00:00:00

Actual result

$ bin/magento indexer:reindex
Design Config Grid index has been rebuilt successfully in 00:00:00
Customer Grid index has been rebuilt successfully in 00:00:00
Product Flat Data index has been rebuilt successfully in 00:00:00
Category Flat Data index has been rebuilt successfully in 00:00:00
Category Products index has been rebuilt successfully in 00:00:00
Product Categories index has been rebuilt successfully in 00:00:00
Product Price index has been rebuilt successfully in 00:00:00
SQLSTATE[23000]: Integrity constraint violation: 1062 Duplicate entry '877-146-1-170-877' for key 'PRIMARY', query was: INSERT INTO `catalog_product_index_eav_temp` (`entity_id`,`attribute_id`,`store_id`,`value`,`source_id`) VALUES ...
Catalog Search index has been rebuilt successfully in 00:00:00
Stock index has been rebuilt successfully in 00:00:00
Catalog Rule Product index has been rebuilt successfully in 00:00:00
Catalog Product Rule index has been rebuilt successfully in 00:00:00
Amasty Customer Group Catalog Rule index has been rebuilt successfully in 00:00:00
Amasty Customer Group Catalog Rule index has been rebuilt successfully in 00:00:00
Amasty Customer Group Catalog Rule index has been rebuilt successfully in 00:00:00
Search Spell-Correction index has been rebuilt successfully in 00:00:00

The solution

I see 2 possible solutions:

• Deduplicate the values while importing the file.
• Add an error while validating the file.

I created a plugin to fix it for now:

<?php

namespace Marissen\MagentoCatalogImportExport\Plugin\Model\Import;

use Magento\CatalogImportExport\Model\Import\Product as Subject;

class Product
{
    public function afterParseMultiselectValues(Subject $subject, array $values): array
    {
        return array_unique($values);
    }
}

[tdgroot]

I added The solution section to get the discussion going, so we can get to a solution faster.

[dmanners]

Hi @tdgroot I think that adding the deduplication of values would be a valid fix in this case.

[dmanners]

Actually talking with @pogster we think that maybe showing this error message would be a good option here.

@pogster if you feel like tackling this at some point feel free to assign it to yourself.

[pogster]

PR is open #117

[dmanners]

Hi, this has been covered in the 2.3-develop branch by the PR https://github.com/magento-engcom/import-export-improvements/pull/117 from @pogster

[tdgroot]

Great work, thanks @pogster and @dmanners!