SpencerTorres
commented
5 days ago I like this idea, and it isn't the first to suggest some form of estimate or progress bar. I've done some local experiments with streaming data, which would include query progress but not an estimate. I feel like most queries are written to be fast, especially for a Grafana use case. I wouldn't want to delay the main query by running an estimate first, or by running an estimate for a known fast query. For long running queries it would make sense to have an estimate. I believe we would want mechanisms in place to support long running queries too, the kind where you can leave it running in the background without having to keep the browser tab open.
In either case, estimates or progress, we don't have a place to render this information. The plugin UI doesn't have much control of these outside panels, and changing information on the query editor itself is usually associated with a state change that re-runs the query.
Perhaps the Grafana team can comment on this idea. Overall I think it's interesting, but we would need to determine when to run the estimate and how to display it
The Grafana BigQuery data source displays a message like "This query will process XXX data" before query execution. A similar message under the query box showing users how much their Clickhouse query will process before the query is run would be very helpful.
This is done in BigQuery by running a Query with the DryRun flag and then rendering the statistics from that DryRun. There is no DryRun concept in Clickhouse, but there is EXPLAIN ESTIMATE - this would at least give us parts/rows/marks.
Per Alexey last year, there was no plan at that time to provide additional cost estimations like bytes read, etc. - this would be great if the Clickhouse team could add on their side at some point (@SpencerTorres)
Happy to hear other thoughts / ideas on how we could surface query impact before execution to Grafana users!