search

cpan-testers / cpantesters-api

An API in to data held by CPAN Testers: Test reports and CPAN uploads
Other
4 stars 4 forks source link

Auto-kill legacy Metabase workers taking too much memory #25

Open preaction opened 6 years ago

preaction commented 6 years ago

The legacy Metabase API (bin/cpantesters-legacy-metabase) starts out at about 70MB resident memory. Once it starts serving requests, it immediately goes to 100MB, and starts growing. I've seen worker processes taking up 2GB of resident memory.

I've reduced the number of requests a single worker can serve before they're killed by the master, and that should help, but it would be better if we could more directly terminate a worker once they go past 400MB or so. By that time, they've spent 10 minutes serving requests and it will be fine to kill them.

The best module to use for this is Mojolicious::Plugin::SizeLimit, but it seems unmaintained. I've left a note on the Github issue saying I will take maintainership (https://github.com/augensalat/mojolicious-plugin-sizelimit/issues/4). If I get no response for a week or so, I should contact them directly. If I get no response after that, I should start looking in to getting COMAINT via PAUSE admins.

This ticket is not to look in to what is causing the memory usage to rise. That is another ticket.

wsdookadr commented 4 years ago

@preaction Please review and try out Mojolicious::Plugin::SizeLimit PR 5