search

unee-t / bugzilla-customisation

Unee-T's customisation on top of Bugzilla
https://hub.docker.com/r/uneet/bugzilla-customisation/
GNU Affero General Public License v3.0
5 stars 7 forks source link

Error sending email - Cannot Allocate memory #42

Closed franck-boullier closed 6 years ago

franck-boullier commented 6 years ago

@kaihendry a weird error happened to some users yesterday.

See the screenshot for more details. Error message was

Couldn't open pipe to sendmail (/usr/sbin/sendmail): Cannot allocate memory

whatsapp image 2018-03-05 at 18 01 22

kaihendry commented 6 years ago

I'm going to assume "Cannot allocate memory" means it ran out of memory on the Bugzilla task.

production-memory

Currently the machines that run Unee-T are t2.mediums with 4GB of RAM.

I've configured the bugzilla instance to allocate at least 1GB, and I have removed the hard limit: https://github.com/unee-t/bugzilla-customisation/commit/e9c82bc9f7136bace0a422c704de356cffa1ba4a

The meteor instance that runs together on the same machine, has the default hard limit of 512mb.

Going forward we could try tweak the mod_perl allocation to be something lower, or just upgrade cluster to t2.large with 8GB of RAM. So a large $0.1168 per Hour from medium $0.0584 per Hour.

kaihendry commented 6 years ago

https://s.natalian.org/2018-03-06/cw-ecs.mp4

I'm going to test a t2.large upgrade in staging first before rolling out to production.

kaihendry commented 6 years ago

Initial approach is to revise down mod_perl memory requirements 800mb -> 400mb. https://github.com/unee-t/bugzilla-customisation/commit/7e4902d944e210b274291fdbaa6748c899425a46

If that doesn't work, I will beef up the instance type.

Also I should try MaxClients https://github.com/mozilla-bteam/bmo/blob/master/conf/httpd.conf#L12-L17

kaihendry commented 6 years ago

I think this bug is dealt with. https://github.com/unee-t/bugzilla-customisation/releases/tag/prod-2018-03-06

We'll reopen is there are any issues. https://ap-southeast-1.console.aws.amazon.com/cloudwatch/home?region=ap-southeast-1#dashboards:name=ECS