search

zabbix / zabbix-docker

Official Zabbix Dockerfiles
https://www.zabbix.com
GNU Affero General Public License v3.0
2.36k stars 1.37k forks source link

Memory leak zabbix server docker 5.0 #871

Closed X4vie3r closed 3 years ago

X4vie3r commented 3 years ago
SUMMARY

Hello,

We have a zabbix-server version 5.0 running on docker and a couple weeks ago we started to see a memory leak on the server. The usage of memory RAM start to grow until the server reach 100% and after that, contunue with a restart of the DB docker.

The database we use is postgres 12 with timescale DB 1.7.5 running with Docker on Centos7.

OS / ENVIRONMENT / Used docker-compose files

Just for your information this is the hardware of the server, and the images that we use for the compose:

CentOS7 Memory RAM:32 GB CPU: 8

zabbix-server: zabbix/zabbix-server-pgsql:centos-5.0.0

zabbix-web-nginx-pgsql: zabbix/zabbix-web-nginx-pgsql:centos-5.0.0

postgres-server: timescale/timescaledb:1.7.5-pg12

CONFIGURATION

The configuration that we use is like this.

.env_db_pgsql (by default)

.env_srv ZBX_STARTPOLLERS=50 ZBX_STARTPOLLERSUNREACHABLE=30 ZBX_STARTTRAPPERS=50 ZBX_STARTPINGERS=50 ZBX_STARTDISCOVERERS=20 ZBX_JAVAGATEWAY_ENABLE=true ZBX_STARTJAVAPOLLERS=5 ZBX_ENABLE_SNMP_TRAPS=true ZBX_CACHESIZE=1G ZBX_HISTORYCACHESIZE=1G ZBX_HISTORYINDEXCACHESIZE=1G ZBX_TRENDCACHESIZE=32M ZBX_VALUECACHESIZE=256M ZBX_LOGSLOWQUERIES=3000

.env_web (by default)

This are the stats of the monitoring platform.

Number of hosts (enabled/disabled/templates) 2451 (180 / 1640 / 631) Number of items (enabled/disabled/not supported) 27488 (18363 / 7702 / 1423) Number of triggers (enabled/disabled [problem/ok]) 8331 3977 / 4354 [54 / 3923]

Please we will apreciate if anyone could help us with this issue and tell us why is this happening, or any solution for this problem that we have.

kind regards,

kernbug commented 3 years ago

It's unclear from description which server (container) is causing memory issues. If Zabbix Server container - could you please show process list output with memory consumption.

At least use 5.0.15 since all known issues are fixed already. Also PgSQL12 with TSDB below 2.0.2 can consume uncontrolled amount of memory for hyperscale partitions:

TSDB issues:

  1. 2874 Fix partitionwise agg crash due to uninitialized memory - fixed in 2.0.2+;

PGSQL issues:

  1. Reduce memory usage for query strings and extension scripts that contain many SQL statements (Amit Langote) https://commitfest.postgresql.org/28/2590/ - fixed in PgSQL 13+;
X4vie3r commented 3 years ago

Hello!

First of all thank you for the quickly response.

The problem that we see is on the Database docker, when we restart it the memory go automatically to 20%, then it keeps growing to 100%.

At least we going to go to upgrade TSDB at version 2.1.0 and see if the problem will dissappear.

kernbug commented 3 years ago

You can use official guide from TimescaleDB to upgrade: https://docs.timescale.com/timescaledb/latest/how-to-guides/update-timescaledb/update-timescaledb-2/#prerequisites