search

Invertus / elasticsearch

Elasticsearch® module for PrestaShop that makes search and filter significantly faster.
47 stars 21 forks source link

Allowed memory size exhausted #12

Open cryptocoinserver opened 6 years ago

cryptocoinserver commented 6 years ago

The environment PrestaShop-Version 1.6.1.15 Elasticsearch version: 5.6.2 Products: 1864 PHP-Version 7.1.3 php_memory_limit: 512M max_execution_time: 20000

This happens, while trying to reindex (both all and missing):

[error] FastCGI sent in stderr: "PHP message: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 20480 bytes) in /var/www/clients/client1/web2/web/classes/db/DbPDO.php on line 154 PHP message: PHP Fatal error: Allowed memory size of 536870912 bytes exhausted (tried to allocate 77824 bytes) in /usr/lib/tideways/Tideways.php on line 354" while reading response header from upstream, client: XXX, server: XXX, request: "POST /adminXXX/index.php?controller=AdminBradSetting&token=XXX HTTP/1.1", upstream: "fastcgi://unix:/var/lib/php5-fpm/web2.sock:", host: "XXX", referrer: "https://XXX/adminXXX/index.php?controller=AdminBradSetting&token=XXX"

I tried:

Replaced some information with XXX for privacy protection.

Thank you for your time and work!

sarjon commented 6 years ago

hi @mysecondworld, what is size of your bulk request? Do your products have a lot of features or attributes?

cryptocoinserver commented 6 years ago

hi @sarjon I tried many different sizes. From 10 to the default. Yes they got many features for example flavours.

The shop i work for, decided to try this modul: https://addons.prestashop.com/en/search-filters/19527-elasticsearchconnector.html And it works very fast and without error. The settings are great, too.

Thank you for your time.

ghost commented 6 years ago

Brad and elasticsearchconnector does not work the same.

I bought the module elasticsaerch and I prefer brad it does not use sql and has filters :1st_place_medal:

veebipoed commented 6 years ago

I try to get it to work for a shop with 12000+ active products, in general the purpose is to use elasticsearch for up to 100000 products. Changing batch size to 10 allowed me to index 408 products before I got also out-of-memory error - so in general I can try to run cron for every 10-15 minutes or write a wrapper to check if the script is running or not.

I haven't investigated how Brad works, but is it possible in general to make it work for ca 20th products? I also need the filtering option, this is why I'm trying out Brad.

It seems bottleneck is the amount of features there is in use in our case. But still some things can be improved from the memory point of view.

nicdnepr commented 5 years ago

Any news? I have shop with 250k goods, run php cli brad cron and get out of memory, I removed this line https://github.com/Invertus/brad/blob/v2/brad.cron.php#L56 How is Bulk request size depends on allocated memory? I try to set Bulk request size much bigger to increase index speed, but not help

nicdnepr commented 5 years ago

PHP Fatal error: Allowed memory size of 6585057280 bytes exhausted (tried to allocate 20480 bytes) in /home/test/public_html/t/classes/db/DbPDO.php on line 154 how can it be?