Operation Insights: Metadata Tags for operations

[erikwrede]

It would be great if Hive reports were taggable and filterable by custom user-defined tags. Use cases for tags could include

• Region (e.g. europe-west, asia-pacific, north-america) for examining perfomance per Region
• Backend-Revision - examining revision-specific performance
• Tenant-name - enabling multitenancy analysis

Essentially, tags complement ClientInfo to provide additional metadata on the request. An operation could have one or multiple tags, resulting in the following changes to the Hive Reporting Spec:

export interface RequestOperation {
  timestamp: number
  operationMapKey: string
  execution: Execution
  metadata?: Metadata
}

export interface Metadata {
  client?: Client
+ tag?: string // OR
+ tags?: string[]
}

In the first iteration, the insights UI could be adjusted to have an additional filter like this:

If you agree on this feature being useful, there's several points to discuss:

• One vs Multiple tags - (utility vs DB index performance)
• Necessity of additional ClickHouse materialized views for Tags (complementing client, operation views)
• Default tag if none was provided (better DB performance), similar to client -> "unknown"

Looking forward to your opinion on this.

Happy to draft an initial PR once we are aligned on the open points.

[kamilkisiela]

Makes sense, it shouldn't be an issue to maintain it and it's relatively easy to implement. Yeah, we could accept such PR.

• Multiple tags (it's better future-wise)
• index for tags in materialized views of ClickHouse

and we're good to go

[erikwrede]

Small update, I still have this on my list and have started work. Some other things sadly got in between, but I'm still hoping to push this through soon!