Increased pid_limit to mattermost container which low value caused vary plugin crashes and unableing to use mmctl inside container
Commentary: In case of our mattermost server I can track by systemd-cgtop command that mattermost container using over 200 tasks. This low value on pid_limit caused vary plugins crashing accidentially and failing container healthcheck. This issues was very hard to investigate. I only discover this when I was needed to use mmctl and I was unable to run binary inside container with next error:
Added memory limitation to mattermost container for preventing potential memory leaks
Commentary: This limitations were added to our mattermost server after several memory leaks during last year. Before upgrading to 7.1.3 several leaks was caused by mention 100+ users over @all command in channel, but this already fixed afaik. Recent leak that was on latest LTS 7.1.3 relase we ain't determined but we guessing it caused by playbooks plugin beacause of a lot of errors on consumtion memory peak.
Summary
Commentary: In case of our mattermost server I can track by
systemd-cgtop
command that mattermost container using over 200 tasks. This low value on pid_limit caused vary plugins crashing accidentially and failing container healthcheck. This issues was very hard to investigate. I only discover this when I was needed to usemmctl
and I was unable to run binary inside container with next error:Commentary: This limitations were added to our mattermost server after several memory leaks during last year. Before upgrading to 7.1.3 several leaks was caused by mention 100+ users over @all command in channel, but this already fixed afaik. Recent leak that was on latest LTS 7.1.3 relase we ain't determined but we guessing it caused by playbooks plugin beacause of a lot of errors on consumtion memory peak.
Hope this information can help someone.