Add all execution stats to the cloud telemetry logging channel

kevin-v-ngo commented 2 years ago

We collect execution statistics and surface them in crdb_internal.statement_statistics. We should also capture these statistics in our cloud telemetry logging channel for query events. A few fields that are missing include:

[x] maxDiskUsage - Maximum temporary disk usage that occurred while executing this statement. This is set in cases where a query had to spill to disk
[x] networkBytes - Number of bytes sent over the network.
[x] bytesRead - Number of bytes read from disk (done as part of https://github.com/cockroachdb/cockroach/pull/85169)
[x] networkMsgs - Number of messages sent over the network
[ ] Retries - Total number of internal retries
[ ] nodes - Ordered list of nodes IDs on which the statement was executed
[x] numRows - Number of rows returned or observed
[x] ovhLat - Difference between svcLat and the sum of parseLat+planLat+runLat latencies
[x] parseLat - Time to transform the SQL string into an abstract syntax tree (AST)
[x] planLat - Time to transform the AST into a logical query plan
[x] rowsRead - number of rows read from disk (done as part of https://github.com/cockroachdb/cockroach/pull/85169)
[x] rowsWritten - number of rows written to disk (done as part of https://github.com/cockroachdb/cockroach/pull/85169)
[x] runLat - time to run the query and fetch or compute the result rows
[x] svcLat - time to service the query, from start of parse to end of execute

The full list is here: https://www.cockroachlabs.com/docs/stable/crdb-internal.html#statistics-column

Jira issue: CRDB-17832

Epic CRDB-32141

kevin-v-ngo commented 1 year ago

Retries would've been helpful to understand workload suffering from contention (write after read, read uncertainty window, closed timestamp).

j82w commented 1 year ago

All MVCC and latency info added in: https://github.com/cockroachdb/cockroach/pull/102820

j82w commented 1 year ago

Nodes needs to wait on: https://github.com/cockroachdb/cockroach/pull/102774

cockroachdb / cockroach

Add all execution stats to the cloud telemetry logging channel #84729