akeneo-labs / EnhancedConnectorBundle

[DEPRECATED] Provides product reader with more option (choice on completeness, choice on enabled, delta based on last export time).
21 stars 11 forks source link

Product Export's readCount is "0" #37

Closed ewallteam closed 8 years ago

ewallteam commented 8 years ago

Hi All, We have used the Akeneo 1.3.28 and imported 12000 products to Akeneo from the Magento without any issues. After that, we have tried to export all product by pim_enhanced_connector but the exported CSV seems empty and log file said that the readcount and writecount are 0

[2015-12-10 04:33:11] batch.DEBUG: Job execution starting: startTime=, endTime=, updatedTime=, status=2, exitStatus=, exitDescription=[], job=[ewall_product_export] [] [] [2015-12-10 04:33:11] batch.INFO: Step execution starting: id=0, name=[pim_enhanced_connector.export.product.title], status=[2], exitCode=[EXECUTING], exitDescription=[], readCount=0, writeCount=0, filterCount=0 [] [] [2015-12-10 04:33:11] batch.DEBUG: Step execution success: id= 674 [] [] [2015-12-10 04:33:11] batch.DEBUG: Step execution complete: id=674, name=[pim_enhanced_connector.export.product.title], status=[1], exitCode=[EXECUTING], exitDescription=[], readCount=0, writeCount=0, filterCount=0 [] [] [2015-12-10 04:33:11] batch.DEBUG: Upgrading JobExecution status: startTime=2015-12-10T04:33:11+00:00, endTime=, updatedTime=, status=3, exitStatus=, exitDescription=[], job=[ewall_product_export] [] []

Help me to find and resolve the issue with product export.

Thanks.

damien-carcel commented 8 years ago

Hello Ewallteam.

I think a very likely issue is a wrong configuration of the export, causing the reader not to read anything. Can you copy/paste a screenshot of your enhanced product export job configuration, please?

Regards, Damien

ewallteam commented 8 years ago

Hi @damien-carcel Sorry for late reply, I have upgraded to my Akeneo to latest version. But still the product export not working even I have using the native CSV connector. When I have run the profile and it was completed suddenly and show the empty summary. See my configuration here https://drive.google.com/file/d/0B8aStV7rYyB6dDc5TXFfV1gxVzA/view?usp=drivesdk

My result is https://drive.google.com/file/d/0B8aStV7rYyB6eWlodXZadDNJYWs/view?usp=drivesdk

damien-carcel commented 8 years ago

Thanks for the data. Looking at your profile configuration, I have an obvious question: you export complete products from the ecommerce channel. Do you have any complete products for this specific channel? If not, it is normal you don't export anything.

Regards, Damien

ewallteam commented 8 years ago

All my products having the 100% completeness. But no one in exported csv.

damien-carcel commented 8 years ago

Another question: are your products categorized? If not, then they cannot be exported, unless you use the last enhanced connector version I tag this morning. It adds a new option to export categorized and/or uncategorized products.

Regards, Damien

ewallteam commented 8 years ago

Hi @damien-carcel

Thanks for your reply. Now its working my products are not categorized that was the problem so I have update the bundle now its working but export was failed due to the below error

Fatal Error: Allowed memory size of 805306368 bytes exhausted

damien-carcel commented 8 years ago

Hello again.

First, I recommend you set PHP memory to 1G. Next, be sure to launch the export in prod mode, not in dev mode (dev mode will cause a memory leak, and so a memory exhaustion).

Finally, there is a memory leak in product exports in Akeneo versions prior to 1.4.9 and 1.3.32, so you need to upgrade to last 1.3 versions to be sure to get read of this issue (you can see the commit on 1.3 here).

ewallteam commented 8 years ago

Thanks @damien-carcel

Now everything working correctly. But I need to move my Akeneo Upgraded files to production server. So please let me know the steps, the moving steps are just copy paste or can I do anything specially?

damien-carcel commented 8 years ago

You can use GIT (or any CVS you are using for your project) to pull the changes on your production server, then upgrading the vendors with composer. When we do not have access to a CVS from the production server, then we usually use Rsync to copy/past changed files.

In any case, don't forget to check the PHP memory on your server, then restart apache and finally clear Symfony cache (php app/console -env=prod cache:clear).

Regards, Damien

ewallteam commented 8 years ago

Hi @damien-carcel My server has hanged while export the products more than 10000. PHP memory limit is 2gb. Any idea?

damien-carcel commented 8 years ago

Hello @ewallteam.

Look at your logs in app/logs/batch_execute.log, there should be more info inside it. What's more, remember that the enhanced connector bundle only manage exports, it is not related to imports at all. To have a better support on this matter, I recommend you report this problem on Akeneo forum.

Regards, Damien

ewallteam commented 8 years ago

Hi @damien-carcel

Sorry for miscommunication I have typed "import" instead "export". Actually I have problem with export. I will check the log files and come back. Moreover I really appreciate your great support. Thanks.

My apache error log shows

[Tue Jan 12 09:40:41 2016] [error] [client 103.242.236.163] PHP Warning: exec(): Unable to fork [ps -p 31992] in /var/www/akeneo_upgraded/akeneo/vendor/akeneo/batch-bundle/Akeneo/Bundle/BatchBundle/Manager/JobExecutionManager.php on line 64, referer:

and My prod.log shows

[2016-01-12 09:42:27] request.CRITICAL: Uncaught PHP Exception PDOException: "SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction" at /var/www/akeneo_upgraded/akeneo/vendor/symfony/symfony/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/PdoSessionHandler.php line 513 {"exception":"[object](PDOException: SQLSTATE[HY000]: General error: 1205 Lock wait timeout exceeded; try restarting transaction at /var/www/akeneo_upgraded/akeneo/vendor/symfony/symfony/src/Symfony/Component/HttpFoundation/Session/Storage/Handler/PdoSessionHandler.php:513)"}

damien-carcel commented 8 years ago

Hello @ewallteam

I encountered this kind of problem before. It is most likely there is another process using your database during the import. Be sure there is no background processes (like completeness calculation or another import/export) running during the import. For large import, it is also better not to edit products during the import. Ideally, large imports should be done during the night, when there is no risk that users do some modifications at the same time.

Regards, Damien

ewallteam commented 8 years ago

Hi @damien-carcel

We want to sync the changes automatically between the Akeneo and Magento though your bundle.Is it possible can you please help me to do the auto sync?

I have created the new ticket for this https://github.com/akeneo-labs/EnhancedConnectorBundle/issues/50

fitn commented 8 years ago

Hi @ewallteam

We could continue the discussion in #50 in this case. Let us know if you encounter again the issue or need more informations :)

Regards, Romain