High memory usage on web browsers for TimeSeries charts

[maxtymchenko]

Describe the bug I'm using an RPI client with GB of RAM as an Chromium/Firefox kiosk dashboard. Adding TimeSeries chart for Zabbix metrics to the dashboard makes both quickly exhausting RAM to the point it completely unresponsive followed by a tab crash.

The issue also reproducible on Windows with Chromium/Firefox/Firefox ESR.

I am aware about old grafana related bugs : https://github.com/grafana/grafana/issues/35987 https://github.com/grafana/grafana/issues/21692 and fix https://github.com/grafana/grafana/pull/36371

but they seems to be unrelated now as I am not able to reproduce it on pure grafana engine.

What I tried:

1. play.grafana.org - Visualizations/ 1 - Time series graphs
2. Internal grafana's "randomwalk" graphs.

In both cases the high RAM consumption issue is not occurring - I am able to run my RPI in kiosk mode for >6 hours with no crashes/RAM growth etc, so it only happens with grafana-zabbix plugin.

Expected behavior No memory leaks happened.

Screenshots N/A at this stage but could be provided if necessary.

Network data Request: {"queries":[{"refId":"A","format":"time_series","datasourceId":2,"rawSql":"SELECT to_char(itemid, 'FM99999999999999999999') AS metric, clock / 30 * 30 AS time, AVG(value) AS value FROM history WHERE itemid IN (37192) AND clock > 1629864410 AND clock < 1629907610 GROUP BY 1, 2 ORDER BY time ASC","maxDataPoints":10000}]}

Response: {"results":{"A":{"frames":[{"schema":{"name":"results","meta":{"executedQueryString":"SELECT to_char(itemid, 'FM99999999999999999999') AS metric, clock / 30 * 30 AS time, AVG(value) AS value FROM history WHERE itemid IN (37192) AND clock \u003e 1629864410 AND clock \u003c 1629907610 GROUP BY 1, 2 ORDER BY time ASC"},"fields":[{"name":"Time","type":"time","typeInfo":{"frame":"time.Time"}},{"name":"37192","type":"number","typeInfo":{"frame":"float64","nullable":true}}]},"data":{"values":[[1629864420000,1629864480000,1629864540000,1629864600000,1629864660000,1629864720000,1629864780000,1629864840000,1629864900000,1629864960000,1629865020000,1629865080000,1629865140000,1629865200000,1629865260000,1629865320000,1629865380000,1629865440000,1629865500000,1629865560000,1629865620000,1629865680000,1629865740000,1629865800000,1629865860000,1629865920000,1629865980000,1629866040000,1629866100000,1629866160000,1629866220000,1629866280000,1629866340000,1629866400000,1629866460000,1629866520000,1629866580000,1629866640000,1629866700000,1629866760000,1629866820000,1629866880000,1629866940000,1629867000000,1629867060000,1629867120000,1629867180000,1629867240000,1629867300000,1629867360000,1629867420000,1629867480000,1629867540000,1629867600000,1629867660000,1629867720000,1629867780000,1629867840000,1629867900000,1629867960000,1629868020000,1629868080000,1629868140000,1629868200000,1629868260000,1629868320000,1629868380000,1629868440000,1629868500000,1629868560000,1629868620000,1629868680000,1629868740000,1629868800000,1629868860000,1629868920000,1629868980000,1629869040000,1629869100000,1629869160000,1629869220000,1629869280000,1629869340000,1629869400000,1629869460000,1629869520000,1629869580000,1629869640000,1629869700000,1629869760000,1629869820000,1629869880000,1629869940000,1629870000000,1629870060000,1629870120000,1629870180000,1629870240000,1629870300000,1629870360000,1629870420000,1629870480000,1629870540000,1629870600000,1629870660000,1629870720000,1629870780000,1629870840000,1629870900000,1629870960000,1629871020000,1629871080000,1629871140000,1629871200000,1629871260000,1629871320000,1629871380000,1629871440000,1629871500000,1629871560000,1629871620000,1629871680000,1629871740000,1629871800000,1629871860000,1629871920000,1629871980000,1629872040000,1629872100000,1629872160000,1629872220000,1629872280000,1629872340000,1629872400000,1629872460000,1629872520000,1629872580000,1629872640000,1629872700000,1629872760000,1629872820000,1629872880000,1629872940000,1629873000000,1629873060000,1629873120000,1629873180000,1629873240000,1629873300000,1629873360000,1629873420000,1629873480000,1629873540000,1629873600000,1629873660000,1629873720000,1629873780000,1629873840000,1629873900000,1629873960000,1629874020000,1629874080000,1629874140000,1629874200000,1629874260000,1629874320000,1629874380000,1629874440000,1629874500000,1629874560000,1629874620000,1629874680000,1629874740000,1629874800000,1629874860000,1629874920000,1629874980000,1629875040000,1629875100000,1629875160000,1629875220000,1629875280000,1629875340000,1629875400000,1629875460000,1629875520000,1629875580000,1629875640000,1629875700000,1629875760000,1629875820000,1629875880000,1629875940000,1629876000000,1629876060000,1629876120000,1629876180000,1629876240000,1629876300000,1629876360000,1629876420000,1629876480000,1629876540000,1629876600000,1629876660000,1629876720000,1629876780000,1629876840000,1629876900000,1629876960000,1629877020000,1629877080000,1629877140000,1629877200000,1629877260000,1629877320000,1629877380000,1629877440000,1629877500000,1629877560000,1629877620000,1629877680000,1629877740000,1629877800000,1629877860000,1629877920000,1629877980000,1629878040000,1629878100000,1629878160000,1629878220000,1629878280000,1629878340000,1629878400000,1629878460000,1629878520000,1629878580000,1629878640000,1629878700000,1629878760000,1629878820000,1629878880000,1629878940000,1629879000000,1629879060000,1629879120000,1629879180000,1629879240000,1629879300000,1629879360000,1629879420000,1629879480000,1629879540000,1629879600000,1629879660000,1629879720000,1629879780000,1629879840000,1629879900000,1629879960000,1629880020000,1629880080000,1629880140000,1629880200000,1629880260000,1629880320000,1629880380000,1629880440000,1629880500000,1629880560000,1629880620000,1629880680000,1629880740000,1629880800000,1629880860000,1629880920000,1629880980000,1629881040000,1629881100000,1629881160000,1629881220000,1629881280000,1629881340000,1629881400000,1629881460000,1629881520000,1629881580000,1629881640000,1629881700000,1629881760000,1629881820000,1629881880000,1629881940000,1629882000000,1629882060000,1629882120000,1629882180000,1629882240000,1629882300000,1629882360000,1629882420000,1629882480000,1629882540000,1629882600000,1629882660000,1629882720000,1629882780000,1629882840000,1629882900000,1629882960000,1629883020000,1629883080000,1629883140000,1629883200000,1629883260000,1629883320000,1629883380000,1629883440000,1629883500000,1629883560000,1629883620000,1629883680000,1629883740000,1629883800000,1629883860000,1629883920000,1629883980000,1629884040000,1629884100000,1629884160000,1629884220000,1629884280000,1629884340000,1629884400000,1629884460000,1629884520000,1629884580000,1629884640000,1629884700000,1629884760000,1629884820000,1629884880000,1629884940000,1629885000000,1629885060000,1629885120000,1629885180000,1629885240000,1629885300000,1629885360000,1629885420000,1629885480000,1629885540000,1629885600000,1629885660000,1629885720000,1629885780000,1629885840000,1629885900000,1629885960000,1629886020000,1629886080000,1629886140000,1629886200000,1629886260000,1629886320000,1629886380000,1629886440000,1629886500000,1629886560000,1629886620000,1629886680000,1629886740000,1629886800000,1629886860000,1629886920000,1629886980000,1629887040000,1629887100000,1629887160000,1629887220000,1629887280000,1629887340000,1629887400000,1629887460000,1629887520000,1629887580000,1629887640000,1629887700000,1629887760000,1629887820000,1629887880000,1629887940000,1629888000000,1629888060000,1629888120000,1629888180000,1629888240000,1629888300000,1629888360000,1629888420000,1629888480000,1629888540000,1629888600000,1629888660000,1629888720000,1629888780000,1629888840000,1629888900000,1629888960000,1629889020000,1629889080000,1629889140000,1629889200000,1629889260000,1629889320000,1629889380000,1629889440000,1629889500000,1629889560000,1629889620000,1629889680000,1629889740000,1629889800000,1629889860000,1629889920000,1629889980000,1629890040000,1629890100000,1629890160000,1629890220000,1629890280000,1629890340000,1629890400000,1629890460000,1629890520000,1629890580000,1629890640000,1629890700000,1629890760000,1629890820000,1629890880000,1629890940000,1629891000000,1629891060000,1629891120000,1629891180000,1629891240000,1629891300000,1629891360000,1629891420000,1629891480000,1629891540000,1629891600000,1629891660000,1629891720000,1629891780000,1629891840000,1629891900000,1629891960000,1629892020000,1629892080000,1629892140000,1629892200000,1629892260000,1629892320000,1629892380000,1629892440000,1629892500000,1629892560000,1629892620000,1629892680000,1629892740000,1629892800000,1629892860000,1629892920000,1629892980000,1629893040000,1629893100000,1629893160000,1629893220000,1629893280000,1629893340000,1629893400000,1629893460000,1629893520000,1629893580000,1629893640000,1629893700000,1629893760000,1629893820000,1629893880000,1629893940000,1629894000000,1629894060000,1629894120000,1629894180000,1629894240000,1629894300000,1629894360000,1629894420000,1629894480000,1629894540000,1629894600000,1629894660000,1629894720000,1629894780000,1629894840000,1629894900000,1629894960000,1629895020000,1629895080000,1629895140000,1629895200000,1629895260000,1629895320000,1629895380000,1629895440000,1629895500000,1629895560000,1629895620000,1629895680000,1629895740000,1629895800000,1629895860000,1629895920000,1629895980000,1629896040000,1629896100000,1629896160000,1629896220000,1629896280000,1629896340000,1629896400000,1629896460000,1629896520000,1629896580000,1629896640000,1629896700000,1629896760000,1629896820000,1629896880000,1629896940000,1629897000000,1629897060000,1629897120000,1629897180000,1629897240000,1629897300000,1629897360000,1629897420000,1629897480000,1629897540000,1629897600000,1629897660000,1629897720000,1629897780000,1629897840000,1629897900000,1629897960000,1629898020000,1629898080000,1629898140000,1629898200000,1629898260000,1629898320000,1629898380000,1629898440000,1629898500000,1629898560000,1629898620000,1629898680000,1629898740000,1629898800000,1629898860000,1629898920000,1629898980000,1629899040000,1629899100000,1629899160000,1629899220000,1629899280000,1629899340000,1629899400000,1629899460000,1629899520000,1629899580000,1629899640000,1629899700000,1629899760000,1629899820000,1629899880000,1629899940000,1629900000000,1629900060000,1629900120000,1629900180000,1629900240000,1629900300000,1629900360000,1629900420000,1629900480000,1629900540000,1629900600000,1629900660000,1629900720000,1629900780000,1629900840000,1629900900000,1629900960000,1629901020000,1629901080000,1629901140000,1629901200000,1629901260000,1629901320000,1629901380000,1629901440000,1629901500000,1629901560000,1629901620000,1629901680000,1629901740000,1629901800000,1629901860000,1629901920000,1629901980000,1629902040000,1629902100000,1629902160000,1629902220000,1629902280000,1629902340000,1629902400000,1629902460000,1629902520000,1629902580000,1629902640000,1629902700000,1629902760000,1629902820000,1629902880000,1629902940000,1629903000000,1629903060000,1629903120000,1629903180000,1629903240000,1629903300000,1629903360000,1629903420000,1629903480000,1629903540000,1629903600000,1629903660000,1629903720000,1629903780000,1629903840000,1629903900000,1629903960000,1629904020000,1629904080000,1629904140000,1629904200000,1629904260000,1629904320000,1629904380000,1629904440000,1629904500000,1629904560000,1629904620000,1629904680000,1629904740000,1629904800000,1629904860000,1629904920000,1629904980000,1629905040000,1629905100000,1629905160000,1629905220000,1629905280000,1629905340000,1629905400000,1629905460000,1629905520000,1629905580000,1629905640000,1629905700000,1629905760000,1629905820000,1629905880000,1629905940000,1629906000000,1629906060000,1629906120000,1629906180000,1629906240000,1629906300000,1629906360000,1629906420000,1629906480000,1629906540000,1629906600000,1629906660000,1629906720000,1629906780000,1629906840000,1629906900000,1629906960000,1629907020000,1629907080000,1629907140000,1629907200000,1629907260000,1629907320000,1629907380000,1629907440000,1629907500000,1629907560000],[0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0,0]]}}]}}}

Software versions

Grafana	Zabbix	Grafana-Zabbix Plugin
8.0.6/8.1.2	5.4.3	4.2.1

[ikogan]

My browser tab crashes very few hours when viewing my dashboards as a result of this.

[gustavoghpf]

I'm having the same problem, after a few hours with an open dashboard the memory usage grows exponentially.

Grafana | Zabbix | Grafana-Zabbix Plugin -- | -- | -- 8.1.2 | 5.0.14 | 4.2.1

[ikogan]

I'm not sure how helpful this will be to folks in this thread but since this memory usage issue has come up twice in the last few months, both in upstream Grafan, and in this plugin, I built a little electron app for kiosk uses that'll just reload the renderer when it crashes: https://github.com/ikogan/grafana-kiosk.

I've only really tested it on an armhf Raspberry Pi and really only for a few hours so far.

[maxtymchenko]

In my case it is crashing every 1-2 hours unfortunately. A little electron app might be a good workaround but I would say I'm interested in issue investigation and resolution as a preferred option :)

[gustavoghpf]

@alexanderzobnin Could you take a look at this issue?

Looking through chrome dev tools, it seems to be easily reproducible. I tested it on two dashboards, one of them being https://play.grafana-zabbix.org/d/000000003/grafana-zabbix-demo?orgId=2&refresh=5s. In tests, memory usage continues to grow until the page crashes

[dprokop]

This issue should be already fixed in 8.1.x by this PR https://github.com/grafana/grafana/pull/38017

[maxtymchenko]

@dprokop thanks for that, however as I see that PR has been merged 19 Aug but the issue was reproducible 26 Aug as I initially reported. as per my initial investigations - I was not able to reproduce it on pure grafana engine but on grafana-zabbix only but I will retry and update closest time.

[maxtymchenko]

@dprokop I've upgraded up-to-date my test kiosk (grafana 8.1.7, zabbix 5.4.5, grafana-zabbix plugin 4.2.1).

Running same test template as I described initially I've got a tab crash in ~30 minutes which confirms the issue still persists unfortunately.

[garrysiberas]

Seeing the same behavior as above. Time series causing memory usage to creep, only with grafana-zabbix plugin.

Grafana 8.3.4, zabbix 5.4.10, grafana-zabbix plugin 4.2.4

[ascendant512]

I've tried testing a variety of configurations after running into this issue:	Grafana	Zabbix plugin	Zabbix server	Leak
8.0.5	4.1.5	5.0.19	no
8.4.6	4.2.5	5.0.19	yes
8.0.5	4.2.5	5.0.19	yes
8.5.0	4.1.5	5.0.19	no
8.5.0	4.2.0	5.0.19	yes

In conclusion, it looks like the plugin version 4.2.0 and newer is associated with the memory leak, regardless of Grafana version. It seems like it was quite a big update, though - it broke several of my timelines in various ways.

[mfedatto]

I'm facing this issue on Grafana 9.0.5

[maxtymchenko]

I gave it a bit more investigation and it looks like zabbix-grafana creates a lot of objects and not cleaning them up which creates a huge memory consumption :

does it make any sense @alexanderzobnin ?

is there anything else I can assist investigating ? is it fixable ?