[QUESTION] Big exports (100k+ cells) fail at store function

[ericr90]

Prerequisites

• [X] Have read and understood: https://github.com/Maatwebsite/Laravel-Excel/blob/3.1/.github/SUPPORT.md
• [X] Checked if your Laravel Excel version is still supported: https://docs.laravel-excel.com/3.1/getting-started/support.html#supported-versions
• [X] Checked that your question isn't already asked before.
• [X] Filled in the entire issue template

Versions

• PHP version: 7.3.12
• Laravel version: 6.18.35
• Package version: 3.1.23

Description

When trying to export big datasets, we encounter memory issues when generating the download file. I tried various options to solve this problem, but I'm out of options. What I've tried:

• fromQuery (so passing the direct query object)
• fromCollection (passing a normal collection as well as lazy collection using cursor())
• Tried using the different memory options in the config file. Ended up in a stalled script that hits the max execution time.

Export goes to csv. I checked memory usage througout the script. All goes well until I call the Excel::store() function. Then memory goes through the roof, ending up in a memory exception. Ini_set tricks would work for small datasets, but it's not a real option for bigger datasets.

What other options do I have?

[patrickbrouwers]

PhpSpreadsheet is known to have difficulties processing very large datasets. When dealing with large datasets and csv, we usually use csv directly. We have a closed source library in the same fashion as this package that only uses csv. (In the simple cases, it should only be a replace of namespace) Feel free to shoot us an e-mail if you would be interested in buying a license.

[stale[bot]]

This bug report has been automatically closed because it has not had recent activity. If this is still an active bug, please comment to reopen. Thank you for your contributions.

[vitorccs]

Hi @ericr90, I had a project which also needed to export huge datasets and was experiencing memory issues.

I created a Laravel project https://github.com/vitorccs/laravel-csv/ which can export huge CSV files, and has been inspired on this awesome library Laravel-Excel, implementing some cool stuffs like FromQuery, WithMapping, WithHeadings, etc.

Hope this can be useful for you or other people.