SigNoz / signoz

SigNoz is an open-source observability platform native to OpenTelemetry with logs, traces and metrics in a single application. An open-source alternative to DataDog, NewRelic, etc. 🔥 🖥. 👉 Open source Application Performance Monitoring (APM) & Observability tool
https://signoz.io
Other
18.66k stars 1.2k forks source link

Enable Transactions OR log group joining #3688

Open ankitnayan opened 11 months ago

ankitnayan commented 11 months ago

https://www.datadoghq.com/blog/logs-transactions-queries/

https://community.splunk.com/t5/Splunk-Search/Joining-two-log-files-that-have-a-common-field/m-p/83747

ankitnayan commented 11 months ago

Joins can be done using clickhouse query though the performance won't be good if enough filtering is not done as join between huge tables will be slow and might throw memory errors in clickhouse.

I tried writing below queries for traces and something similar for logs should also work.

  1. Table Panel
    
    WITH A AS
    (
    SELECT traceID, durationNano
    FROM signoz_traces.distributed_signoz_index_v2
    WHERE ((stringTagMap['param.location']) = '728,326') AND (timestamp > (now() - toIntervalMinute(30)))
    ), B AS 
    (
    select traceID, serviceName FROM signoz_traces.distributed_signoz_index_v2 WHERE timestamp > (now() - toIntervalMinute(30))
    )

SELECT avg(A.durationNano), B.serviceName from A INNER JOIN B ON A.traceID=B.traceID group by B.serviceName;


2. Timeseries Panel

WITH A AS ( SELECT traceID, durationNano, timestamp FROM signoz_traces.distributed_signoz_index_v2 WHERE ((stringTagMap['param.location']) = '728,326') AND (timestamp > (now() - toIntervalMinute(30))) ), B AS ( select traceID, serviceName FROM signoz_traces.distributed_signoz_index_v2 WHERE timestamp > (now() - toIntervalMinute(30)) )

SELECT count(A.traceID), B.serviceName, toStartOfInterval(A.timestamp, toIntervalMinute(1)) AS interval from A INNER JOIN B ON A.traceID=B.traceID group by (interval, B.serviceName);