grafana / grafana

The open and composable observability and data visualization platform. Visualize metrics, logs, and traces from multiple sources like Prometheus, Loki, Elasticsearch, InfluxDB, Postgres and many more.
https://grafana.com
GNU Affero General Public License v3.0
61.85k stars 11.79k forks source link

Make timeout of /api/reports/render/pdfs configurable #82634

Open gelicia opened 5 months ago

gelicia commented 5 months ago

Discussed in https://github.com/grafana/grafana/discussions/69336

Originally posted by **zhichli** May 31, 2023 We have an nginx proxy in front of grafana pro server with HTTP request timeout set at 200 seconds, so when users access the `/api/reports/render/pdfs` to render a pdf for a dashboard, if the HTTP request takes longer than the timeout, the nginx would return 504 timeout to the users. We have this timeout configured for intentional business and performance reasons. However, the issue is that the Grafana server would not be aware of the proxy timeout and continue processing the rendering request by talking to the image renderer and getting the entire pdf completely rendered even if the nginx proxy timed out already. This continuation is a problem since it basically waste CPU and memory capacity on the host machine even though the users would never get the rendered results. The feature request is to add configurable (e.g. in `ini`) timeout for reports api (i.e. `/api/reports/render/pdfs`) for us to better control the report flow.
AgnesToulet commented 5 months ago

Hi! We are refactoring the PDF feature to make only one call to the image renderer to generate a PDF. This should make the feature much faster and take less than 200s to create a PDF.

Also, we already have a configurable timeout in the reporting section. For now, it's used to timeout individual image renderer requests, though, so the whole feature can take much longer. After the refactoring, as there will be only one request, it will work as this feature request asks.