search

in2code-de / powermail

This is the official repository of the TYPO3 extension powermail! Powermail is a well-known, editor-friendly, powerful and easy mailform extension for TYPO3
https://in2code.de
88 stars 174 forks source link

Backend-module: CSV export timeout #1006

Open tchevalier opened 7 months ago

tchevalier commented 7 months ago

PHP: 8.2.14 Typo3: v12.4.12 Powermail: v12.2.1

Hello, We are trying to export the full mailing list (around 900 rows) but we've got a 500 error on production environment. It looks like a timeout issue. We have no issue on local environment with same data. Debugging the export, I figure out that the problem is on the template rendering. I think it would be a way more efficient to use a PHP algo and render the file using render($myCsvString) (instead of render() that calculates Resources/Private/Templates/Module/ExportCsv.html). Is it something in your pipe?

Thank you, Thibaut

mschwemer commented 7 months ago

Hi

We got the same issue from a customer lately.

As far as I found out, it is a memory issue (at least in our case). If there are very many mails to export, the php_memory_limit is nearly reached. Since the answers are fetched via lazy loading, the memory is filled up quickly.

In our case only 15MB are left for loading the answers. The memory limit is reached after 25 mails.

In order to fix this, we need to rework the database part in the exports completely.

Update

I think, the initial query must be split in smaller trunks and the rendering adapted to it. We will need a feature toggle for it, in order not to break existing installations (if somebody modified the ExportCsv template).

Marcus

BTW: Same issue exists with xlsx export.