grafana / grafana-image-renderer

A Grafana backend plugin that handles rendering of panels & dashboards to PNGs using headless browser (Chromium/Chrome)
Apache License 2.0
375 stars 151 forks source link

Chrome Processes Never Killed - Increasing Memory Profile #407

Open Evesy opened 1 year ago

Evesy commented 1 year ago

What happened: Headless chrome processes spawned by image render plugin are not killed after the image has been rendered. This resulted in memory growth of Grafana every time an image render was requested, eventually resulting in the container running Grafana running out of memory

What you expected to happen: Headless chrome processes should be killed when finished with

How to reproduce it (as minimally and precisely as possible):

Above shows the memory profile of our Grafana instance, the jumps in memory directly correlate with bulk numbers of image renders being made. Jumping into the container there were over 400 chrome processes running

Anything else we need to know?:

Environment:

joanlopez commented 1 year ago

Hi @Evesy,

Thanks for reporting this issue. I definitely think it would be great to analyze the situation you described, but as the recommended way to run the Image Renderer in production is as a standalone service (either Node or as a Docker container), and we haven't seen any memory leak and/or OOM in such case, thus it looks plugin-specific, we're going to consider this very low priority.

However, this is an open source project, so just feel free to take this as an opportunity to contribute to the project.

Thanks!