akeneo / pim-community-dev

[Community Development Repository] The open source Product Information Management (PIM)
http://www.akeneo.com
Other
954 stars 516 forks source link

Allowed memory size exhausted on classic csv products import #11929

Closed simoncarre closed 4 years ago

simoncarre commented 4 years ago

Hi!

I have memory issues during large csv products import:

PHP Fatal error:  Allowed memory size of 1073741824 bytes exhausted (tried to allocate 8388608 bytes) in .../vendor/symfony/symfony/src/Symfony/Component/Debug/DebugClassLoader.php on line 183

What is the tip for importing important CSV files? Do you have to manually cut the files?

PHP Version : 7.2.29 PHP Memory limit : 1024MB Akeneo Version : CE 3.2.46

jmleroux commented 4 years ago

Hello @simoncarre ,

I see a DebugClassLoader reference. Are you working in prod mode?

simoncarre commented 4 years ago

Hello @jmleroux,

Thank you for quick response. I created a preprod environment which is a copy of theprod environment (copy of app/config/config_prod.yml to app/config/config_preprod.yml). My queue command running is : /usr/bin/php /var/www/my-project/current/bin/console akeneo:batch:job-queue-consumer-daemon --env=preprod --no-debug

jmleroux commented 4 years ago

Is it possible to test in prod mode or do you have to much difference?

simoncarre commented 4 years ago

I tested in prod mode and it actually works without issues. Thank you! I was missing the environment variable SYMFONY_DEBUG for the preprod environment :

File bin/console :

$debug = getenv('SYMFONY_DEBUG') !== '0' && !$input->hasParameterOption(['--no-debug', '']) && $env !== 'prod' && false === $behat;