Altinity / clickhouse-grafana

Altinity Grafana datasource plugin for ClickHouse®
MIT License
713 stars 120 forks source link

Support for multiple grouping columns for calculations like increaseColumns #520

Closed JustinMason closed 6 months ago

JustinMason commented 6 months ago

Increase Columns only supports one column as the grouping option.

I have metrics from the ClickHouse Otel Exporter. https://github.com/open-telemetry/opentelemetry-collector-contrib/tree/main/exporter/clickhouseexporter

I'd like to be able to run increase calculations that support multiple grouping columns, because the metrics typically have multiple attributes that require distinct groupings for correct increase calculations.

Your example statement uses runningDifference, which Calculates the difference between two consecutive row values in the data block.

SELECT
    t,
    groupArray((increaseColumns, max_0_Increase)) AS groupArr
FROM
(
    SELECT
        t,
        Protocol,
        if (runningDifference(max_0) < 0 OR neighbor(increaseColumns,-1,increaseColumns) != increaseColumns, 0, runningDifference(max_0)) AS max_0_Increase
    FROM
    (
        SELECT
            (intDiv(toUInt32(EventTime), 60) * 60) * 1000 AS t,
            Protocol AS increaseColumns,
            max(Requests) AS max_0
        FROM requests
        WHERE ((EventDate >= toDate(1535711819)) AND (EventDate <= toDate(1535714715)))
        AND ((EventTime >= toDateTime(1535711819)) AND (EventTime <= toDateTime(1535714715)))
        AND (Protocol IN ('udp', 'tcp'))
        GROUP BY
            t,
            Protocol
        ORDER BY
            t,
            Protocol
    )
)
GROUP BY t
ORDER BY t

How can I group by additional columns from the data.

If my table was:

CREATE TABLE
(
   EventTime,
   Requests,
   Protocol,
   IsSecureRequest
)

How could I get distinct increases for Protocol, IsSecureRequest The existing query won't distinguish between the second grouping IsSecureRequest. Ideally I would have multiple columns that need to be grouped into Increases. The current approach won't support this.

I tried expanding IncreaseColumns: (Protocol, IsSecureRequest) AS increaseColumns

Sense the "the difference between two consecutive rows" doesn't understand each grouping is its own logical set of rows the end result merges the two instead of groups by them.

Slach commented 6 months ago

try to use $increaseColumns(concat(Protocol,if(isSecureRequest,"_secure","") AS increaseColumns) and report if will success for you

we will have one data block for runningDifference cause use sub queries