search

magento / data-migration-tool

Magento Data Migration Tool
https://devdocs.magento.com/guides/v2.3/migration/bk-migration-guide.html
Open Software License 3.0
336 stars 200 forks source link

Mismatch of entities in the document: inventory_source_item Source: 2316 Destination: 1 #640

Closed benhtn closed 5 years ago

benhtn commented 5 years ago

I'm on CentOS 7, PHP 7.2.13 using, nginx version: nginx/1.12.2, MySQL 5.7.23.

I've got a live CE 1.9.3.6 store and am importing settings and data using the data import tool into a CE 2.3.0 dev environment.

My import commands looked like this: 

bin/magento migrate:settings -r -a /usr/share/nginx/html/magento2/vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.6/config.xml
bin/magento migrate:data -r -a /usr/share/nginx/html/magento2/vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.6/config.xml

During initial import I got these errors: 

[2019-01-05 16:02:05][WARNING]: Source documents are not mapped: le_cartmigration_imports,le_cartmigration_recent,le_cartmigration_user,sagepayreporting_fraud,sagepaysuite_action,sagepaysuite_debug,sagepaysuite_fraud,sagepaysuite_paypaltransaction,sagepaysuite_session,sagepaysuite_tokencard,sagepaysuite_transaction,sagepaysuite_transaction_queue,searchanise_config,searchanise_queue 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_order. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_order_item. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_order_payment. Fields: repeat_code 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_quote_address. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_quote_address_item. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_quote_item. Fields: base_surcharge_amount,surcharge_amount 
[2019-01-05 16:02:05][WARNING]: Source fields are not mapped. Document: sales_flat_quote_payment. Fields: sagepay_token_cc_id,repeat_code

So I added this to vendor/magento/data-migration-tool/etc/opensource-to-opensource/1.9.3.6/map.xml in the source > document_rules section:

<ignore><document>le_cartmigration_imports</document></ignore>
<ignore><document>le_cartmigration_recent</document></ignore>
<ignore><document>le_cartmigration_user</document></ignore>
<ignore><document>sagepayreporting_fraud</document></ignore>
<ignore><document>sagepaysuite_action</document></ignore>
<ignore><document>sagepaysuite_debug</document></ignore>
<ignore><document>sagepaysuite_fraud</document></ignore>
<ignore><document>sagepaysuite_paypaltransaction</document></ignore>
<ignore><document>sagepaysuite_session</document></ignore>
<ignore><document>sagepaysuite_tokencard</document></ignore>
<ignore><document>sagepaysuite_transaction</document></ignore>
<ignore><document>sagepaysuite_transaction_queue</document></ignore>
<ignore><document>searchanise_config</document></ignore>
<ignore><document>searchanise_queue</document></ignore>

and this to the source > field_rules section:

<ignore><field>sales_flat_order.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_order.surcharge_amount</field></ignore>
<ignore><field>sales_flat_order_item.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_order_item.surcharge_amount</field></ignore>
<ignore><field>sales_flat_order_payment.repeat_code</field></ignore>
<ignore><field>sales_flat_quote_address.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_address.surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_address_item.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_address_item.surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_item.base_surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_item.surcharge_amount</field></ignore>
<ignore><field>sales_flat_quote_payment.sagepay_token_cc_id</field></ignore>
<ignore><field>sales_flat_quote_payment.repeat_code</field></ignore>

And pointed config.xml to the edited map.xml.

Rerunning the migration tool, into a completely reset and clean destination 2.3.0 store/ db, I get the following warnings:

[2019-01-05 18:03:12][WARNING]: Mismatch of entities in the document: customer_visitor Source: 705985 Destination: 705982
[2019-01-05 18:03:12][WARNING]: Volume Check failed
...
[2019-01-05 18:03:15][WARNING]: Mismatch of entities in the document: inventory_source_item Source: 2316 Destination: 1
[2019-01-05 18:03:15][WARNING]: Volume Check failed

The troubleshooting guide doesn't have much to say on this, other than using the delta option. https://devdocs.magento.com/guides/v2.1/migration/migration-troubleshooting.html

Using the delta option I then get:

[2019-01-05 18:04:26][WARNING]: Mismatch of entities in the document: catalog_product_entity_varchar Source: 30938 Destination: 30130
[2019-01-05 18:04:26][WARNING]: Mismatch of entities in the document: session Source: 32371 Destination: 32327
[2019-01-05 18:04:26][WARNING]: Volume Check failed

[2019-01-05 18:04:28][WARNING]: Mismatch of entities in the document: inventory_source_item Source: 2316 Destination: 1
[2019-01-05 18:04:28][WARNING]: Volume Check failed

catalog_product_entity_varchar and inventory_source_item values don't change in source nor destination, sessions increase in the source, but not the detination.

When I visit the new site all the product categories say "We can't find products matching the selection." and in the admin backend I can see all the imported categories and see the "Products In Category" listed OK apparently, but when I try to go to Catalog > Products I get a 500 error and in the PHP error log I see:

2019/01/05 18:10:47 [error] 1165#0: *784 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Method Magento\Ui\TemplateEngine\Xhtml\Result::__toString() must not throw an exception, caught TypeError: Argument 1 passed to Magento\InventorySalesAdminUi\Model\GetSalableQuantityDataBySku::execute() must be of the type string, null given, called in /usr/share/nginx/html/magento2/vendor/magento/module-inventory-sales-admin-ui/Ui/Component/Listing/Column/SalableQuantity.php on line 70 in /usr/share/nginx/html/magento2/vendor/magento/module-ui/Component/Wrapper/UiComponent.php on line 0" while reading response header from upstream, client: 192.168.1.91, server: lws2.local, request: "GET /admin_mvea3p/catalog/product/index/key/40476ba4d69578fbcff70d90ac51cda930730b71e3db65430d327cdbe40a32ce/ HTTP/1.1", upstream: "fastcgi://unix:/run/php-fpm/php-fpm.sock:", host: "lws2.local", referrer: "http://lws2.local/admin_mvea3p/catalog/category/edit/key/c1e6582aa7733d99bf1d2a4b4883728be00e92ab4f33bf43d82bbfdc7d7d67f3/id/4/"

I am expecting to have all products in the destination database and to be able to view them on front and backends.

I see from this issue https://github.com/magento/data-migration-tool/issues/628 that perhaps SKUs are required and if I run this query 

SELECT entity_id as product_id, sku FROM catalog_product_entity

on my source db I see SKU is NULL in all cases. Is having SKUs on all products a requirement of the tool?

victor-v-rad commented 5 years ago

SKU is required field when you create new product in Admin Panel. Maybe some of your extensions override it somehow. I tried to set null SKU in catalog_product_entity and got error when open product in admin panel or fronted.

charmming commented 5 years ago

please anybody help me...during migration i got this error and after migration opening all customer page shows the second error. thank you.

[2019-06-18 03:46:13][WARNING]: Mismatch of entities in the document: inventory_source_item Source: 7535 Destination: 7203 [2019-06-18 03:46:13][WARNING]: Volume Check failed

{"0":"Type Error occurred when creating object: Magento\Eav\Model\Entity\Attribute\Source\Config","1":"

1 Magento\Framework\ObjectManager\Factory\Compiled->create('Magento\Eav\Mode...', array()) called at [vendor/magento/framework/ObjectManager/ObjectManager.php:56]\n#2 Magento\Framework\ObjectManager\ObjectManager->create('Magento\Eav\Mode...', array()) called at [vendor/magento/framework/Validator/UniversalFactory.php:36]\n#3 Magento\Framework\Validator\UniversalFactory->create('Magento\Eav\Mode...') called at [vendor/magento/module-eav/Model/Entity/Attribute/AbstractAttribute.php:641]\n#4 Magento\Eav\Model\Entity\Attribute\AbstractAttribute->getSource() called at [vendor/magento/module-customer/Model/AttributeMetadataConverter.php:68]\n#5 Magento\Customer\Model\AttributeMetadataConverter->createMetadataAttribute(&Magento\Customer\Model\Attribute#0000000067aa633c000000002b7d916f#) called at [vendor/magento/module-customer/Model/Metadata/CustomerMetadata.php:87]\n#6 Magento\Customer\Model\Metadata\CustomerMetadata->getAttributeMetadata('customerstatus') called at [vendor/magento/module-customer/Model/Metadata/CustomerMetadata.php:119]\n#7 Magento\Customer\Model\Metadata\CustomerMetadata->getAllAttributesMetadata() called at [vendor/magento/module-customer/Model/Metadata/CachedMetadata.php:85]\n#8 Magento\Customer\Model\Metadata\CachedMetadata->getAllAttributesMetadata() called at [vendor/magento/module-customer/Ui/Component/Listing/AttributeRepository.php:78]\n#9 Magento\Customer\Ui\Component\Listing\AttributeRepository->getList() called at [vendor/magento/module-customer/Ui/Component/Listing/Columns.php:102]\n#10 Magento\Customer\Ui\Component\Listing\Columns->prepare() called at [vendor/magento/framework/View/Layout/Generator/UiComponent.php:164]\n#11 Magento\Framework\View\Layout\Generator\UiComponent->prepareComponent(&Magento\Customer\Ui\Component\Listing\Columns#0000000067aa67ff000000002b7d916f#) called at [vendor/magento/framework/View/Layout/Generator/UiComponent.php:161]\n#12 Magento\Framework\View\Layout\Generator\UiComponent->prepareComponent(&Magento\Ui\Component\Listing#0000000067aa67fe000000002b7d916f#) called at [vendor/magento/framework/View/Layout/Generator/UiComponent.php:142]\n#13 Magento\Framework\View\Layout\Generator\UiComponent->generateComponent(&Magento\Framework\View\Layout\Data\Structure#0000000067aa6e7f000000002b7d916f#, 'customer_listing', array('attributes' => array('group' => '', 'component' => '', 'aclResource' => '', 'visibilityCondit...' => array())), &Magento\Framework\View\Layout\Interceptor#0000000067aa6e86000000002b7d916f#) called at [vendor/magento/framework/View/Layout/Generator/UiComponent.php:102]\n#14 Magento\Framework\View\Layout\Generator\UiComponent->process(&Magento\Framework\View\Layout\Reader\Context#0000000067aa6501000000002b7d916f#, &Magento\Framework\View\Layout\Generator\Context#0000000067aa673f000000002b7d916f#) called at [vendor/magento/framework/View/Layout/GeneratorPool.php:81]\n#15 Magento\Framework\View\Layout\GeneratorPool->process(&Magento\Framework\View\Layout\Reader\Context#0000000067aa6501000000002b7d916f#, &Magento\Framework\View\Layout\Generator\Context#0000000067aa673f000000002b7d916f#) called at [vendor/magento/framework/View/Layout.php:343]\n#16 Magento\Framework\View\Layout->generateElements() called at [vendor/magento/framework/View/Layout/Builder.php:129]\n#17 Magento\Framework\View\Layout\Builder->generateLayoutBlocks() called at [vendor/magento/framework/View/Page/Builder.php:55]\n#18 Magento\Framework\View\Page\Builder->generateLayoutBlocks() called at [vendor/magento/framework/View/Layout/Builder.php:65]\n#19 Magento\Framework\View\Layout\Builder->build() called at [vendor/magento/framework/View/Layout.php:253]\n#20 Magento\Framework\View\Layout->build() called at [vendor/magento/framework/View/Layout.php:875]\n#21 Magento\Framework\View\Layout->getBlock('menu') called at [vendor/magento/module-backend/Model/View/Result/Page.php:26]\n#22 Magento\Backend\Model\View\Result\Page->setActiveMenu('Magento_Customer...') called at [vendor/magento/module-customer/Controller/Adminhtml/Index/Index.php:28]\n#23 Magento\Customer\Controller\Adminhtml\Index\Index->execute() called at [vendor/magento/framework/App/Action/Action.php:108]\n#24 Magento\Framework\App\Action\Action->dispatch(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/module-backend/App/AbstractAction.php:231]\n#25 Magento\Backend\App\AbstractAction->dispatch(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:58]\n#26 Magento\Customer\Controller\Adminhtml\Index\Index\Interceptor->callParent('dispatch', array(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#)) called at [vendor/magento/framework/Interception/Interceptor.php:138]\n#27 Magento\Customer\Controller\Adminhtml\Index\Index\Interceptor->Magento\Framework\Interception{closure}(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/module-backend/App/Action/Plugin/Authentication.php:143]\n#28 Magento\Backend\App\Action\Plugin\Authentication->aroundDispatch(&Magento\Customer\Controller\Adminhtml\Index\Index\Interceptor#0000000067aa61eb000000002b7d916f#, &Closure#0000000067aa61e5000000002b7d916f#, &Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:135]\n#29 Magento\Customer\Controller\Adminhtml\Index\Index\Interceptor->Magento\Framework\Interception{closure}(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:153]\n#30 Magento\Customer\Controller\Adminhtml\Index\Index\Interceptor->callPlugins('dispatch', array(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#), NULL) called at [generated/code/Magento/Customer/Controller/Adminhtml/Index/Index/Interceptor.php:26]\n#31 Magento\Customer\Controller\Adminhtml\Index\Index\Interceptor->dispatch(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/App/FrontController.php:159]\n#32 Magento\Framework\App\FrontController->processRequest(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#, &Magento\Customer\Controller\Adminhtml\Index\Index\Interceptor#0000000067aa61eb000000002b7d916f#) called at [vendor/magento/framework/App/FrontController.php:99]\n#33 Magento\Framework\App\FrontController->dispatch(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:58]\n#34 Magento\Framework\App\FrontController\Interceptor->_callParent('dispatch', array(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#)) called at [vendor/magento/framework/Interception/Interceptor.php:138]\n#35 Magento\Framework\App\FrontController\Interceptor->Magento\Framework\Interception{closure}(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/Interception/Interceptor.php:153]\n#36 Magento\Framework\App\FrontController\Interceptor->callPlugins('dispatch', array(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#), array(array('default_store_se...', 'page_cache_from...', 'storeCookieValid...', 'install', 'configHash'))) called at [generated/code/Magento/Framework/App/FrontController/Interceptor.php:26]\n#37 Magento\Framework\App\FrontController\Interceptor->dispatch(&Magento\Framework\App\Request\Http#0000000067aa66ff000000002b7d916f#) called at [vendor/magento/framework/App/Http.php:136]\n#38 Magento\Framework\App\Http->launch() called at [vendor/magento/framework/App/Bootstrap.php:258]\n#39 Magento\Framework\App\Bootstrap->run(&Magento\Framework\App\Http\Interceptor#0000000067aa6615000000002b7d916f#) called at [index.php:39]\n","url":"/admin/customer/index/index/key/d1ee2aeb7f01d80b4dad604c2a131e393fd26935767a2c252777912e7c814828/","script_name":"/index.php"}root@mail:/var/www/html/magento2/var/report

ninjacoder30 commented 5 years ago

@charmming even i am getting the same error while data-migration

Ddcdom commented 4 years ago

@victor-v-rad I know this is old, but its a bug in M1 which allows you to create a product without an SKU. If you open an existing product, click 'duplicate', then it creates a new product in the database without an SKU. So if you duplicate a product, then close the page without setting an SKU, you get an entry in catalog_product_entity with a NULL for SKU, which then causes the volume check to fail on upgrade.

neha023 commented 3 years ago

Anyone found solution of this? As I am facing issue due to duplicate products with empty SKU in M1's DB