search

aerni / statamic-advanced-seo

Comprehensive SEO addon for Statamic with flexibility in mind
https://statamic.com/addons/aerni/advanced-seo
10 stars 5 forks source link

Improve Sitemap generation on larger sites with small memory limit (128M) #122

Closed tobimori closed 5 months ago

tobimori commented 5 months ago

Currently, when rendering the sitemap on our Statamci site with a default memory limit of 128M and around 450 pages across all languages in two collections, all requests fail with a memory limit error.

2024/01/16 14:06:57 [error] 915416#915416: *709626 FastCGI sent in stderr: "PHP message: PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 20480 bytes) in /home/enfore-vnt2l/next.enfore.com/vendor/laravel/framework/src/Illuminate/Cache/FileStore.php on line 310; PHP message: PHP Fatal error:  Allowed memory size of 134217728 bytes exhausted (tried to allocate 32768 bytes) in /home/enfore-vnt2l/next.enfore.com/vendor/symfony/error-handler/Error/FatalError.php on line 1" while reading response header from upstream, client: 64.252.88.246, server: next.enfore.com, request: "GET /sitemap.xml HTTP/1.1", upstream: "fastcgi://unix:/run/php/php8.3-fpm-enfore-vnt2l.sock:", host: "www.enfore.com"

It'd be great if there was a way for sitemap generation to be "off loaded" in the background (e.g. in a queue worker on changes) instead of being on demand so we can avoid setting a high memory limit.

aerni commented 5 months ago

Thanks for bringing this to my attention. The sitemaps are quite intensive as they are querying and filtering a bunch of data. Offloading to a queue sounds like a feasible feature request. For now, I suggest increasing the memory limit.