Problem "Allowed memory size"

[pcmanprogrammeur]

Hi all, I am using the bundle to generate big feeds (1000000 items) and i have a problem of memory. The code looks like this :

        $responseView = new ResponseView();
        $data = [];
        foreach ($items as $item) {
            $data[] = $this->adMapper->getViewData($item, $conf); // Memory error
        }
        $responseView->setCode(Response::HTTP_OK);
        $responseView->setMessage('OK');
        $responseView->setData($data);

        return View::create($responseView, $responseView->getCode());

Is there a solution to create big feeds please ? Is this bundle appropriated to this use case please ?

Regards.

[GuilhemN]

Well since you wrap your data in a custom object ResponseView it should not be moved around too much and the issue probably is not with this bundle but rather with the memory management of PHP.

You could try using a generator for your data instead of creating it all at once but I don't know too what extend it would help but you can try. If you don't actually use all the data given by $this->adMapper->getViewData in the serialized output (if you have a custom normalizer for instance) you could try to eliminate it lazily (with a generator again) to reduce the memory consumption.

[pcmanprogrammeur]

Hi, I tried your solution :

$ads = $adsProvider->getAllAds(); // Generator object
$view = View::create();
$view->setData($ads);
return $view;

But the view is not generated step by step. I wait all the feed to be generated before receiving the first byte.

[InjustFr]

As @GuilhemN said, this has more to do with your logic than the bundle itself. 1 000 000 items is a LOT. Maybe add some pagination mechanism ?

[GuilhemN]

Indeed, this bundle does not support streamed responses. I don't think there exists any easy way to stream responses with templates, or json responses in symfony actually.

But as @InjustFr said, some pagination would probably be a better fix for your case. I'm thus closing this as won't fix.