avstudnitz / AvS_FastSimpleImport

Wrapper for Magento ImportExport functionality, which imports products and customers from arrays
308 stars 146 forks source link

memory size exhausted #267

Open lucop1 opened 8 years ago

lucop1 commented 8 years ago

Hi, I've a problem with the plugin.

I've to import a lot of products from an XML file.

I'm running a foreach loop to then I create the $data array with all the products values, then foreach product I run the import function:

try {
    /** @var $import AvS_FastSimpleImport_Model_Import */
    $import = Mage::getModel('fastsimpleimport/import');
    $import->setPartialIndexing(false)->processProductImport($data);
} catch (Exception $e) {
}

but I see from the console that every time a product has been imported the memory usage of php increase a lot by 10-20 MB at time. And after 10-15 products it shows the default php memory limit error.

PHP Fatal error:  Allowed memory size of 536870912 bytes exhausted (tried to allocate 64 bytes) in /home/xxxxx/public_html/lib/Varien/Db/Adapter/Pdo/Mysql.php on line 2939

This is an example of a product I array I use for configurable products:

$data[] = array(
                'name' => (string)$item->NAME,        // name
                'sku' => (string)$item->ID,        // sku
                '_type' => 'simple',
                'special_price' => $n_price,    // special price
                'price' => $price,    // price
                'costo' => replace_vir($item->PRICE_RES),    // price
                'price_min' => $prezzo,    // price
                '_attribute_set' => $attribute_set,            // attribute_set
                '_product_websites' => 'base',
                'description' => (string)$item->DESCRIPTION_IT,        // full description
                'short_description' => taglia_parola((string)$item->DESCRIPTION_IT, 245),    // short description
                'qty' => $qty,        // qty
                '_category' => $cats,               // ID of categories
                $taglia => (string)$item->SIZE,
                'ext_id' => (string)$item->BARCODE,
                'visibility' => 1,
                //'weight' => (string)$item->mass,        // weight
                //'tax_class_id' => '2',                    // tax class id (check your ids)
                //'manufacturer' => (string)$manufacturer,        // manufacturer
                'meta_title' => (string)$item->NAME,            // meta title
                'meta_description' => taglia_parola((string)$item->DESCRIPTION_IT, 145),
                //'color' => $color,
                'status' => $sstatus,
                'is_in_stock' => $status,
                'disponibile'=>$disp
            );

 $data[] = array(
            'sku' => $sku,
            '_type' => 'configurable',
            '_attribute_set' => $attribute_set,
            '_product_websites' => 'base',
            'price' => $price,
            'special_price' => $n_price,
            'costo' => $costo,    // price
            'price_min' => $price_min,    // price
            'name' => $name,
            'description' => $description,
            'short_description' => taglia_parola($short_description, 245),
            'status' => $cstatus,
            'visibility' => 4,
            'tax_class_id' => 2,
            'is_in_stock' => $status,
            '_category' => $cats,
            'meta_title' => $name,            // meta title
            'meta_description' => taglia_parola($short_description, 145),
            '_super_products_sku' => $configurables[$i],
            '_super_attribute_code' => $taglia,

        );

How can I solve this problem ?

Thank you very much for the support!

sylvainraye commented 8 years ago

And why not use a bulk of products to import? Instead all in one ?

avstudnitz commented 8 years ago

10-15 is much too few to reach any memory limits. I have done 50k in single runs with 1G of memory. There must be something else what's wrong, maybe the routine to retrieve the array.

paales commented 8 years ago

Please make sure you are not calling processProductImport in a loop.

jzahedieh commented 8 years ago

How many configurable products do you have in the system? I ran into a memory issue, this is fixed by PR https://github.com/avstudnitz/AvS_FastSimpleImport/pull/181