search

biigle / reports

:m: BIIGLE module to generate reports for projects and volumes
GNU General Public License v3.0
1 stars 2 forks source link

Out of memory error with CSV report #127

Open mzur opened 1 month ago

mzur commented 1 month ago

There was an out of memory error (1 GB) with the CSV report 17425 (for project 2704). It is a report for about 500k annotations. The CSV report should be able to handle this, so investigate where the problem is.

mzur commented 1 month ago

It turned out that also a lot of freehand polygons were involved in the failing report, so this may be part of the problem.

mzur commented 4 days ago

The reason is simple but the solution not so much. The report CSV is not generated iteratively. It fetches all DB rows at once. To fix this bug, change the code to work iteratively instead.