resource-disaggregation / snowset

Snowflake dataset containing statistics for 70 million queries over 14 day period
103 stars 22 forks source link

Tools for snowflake query analysis ? #5

Open rollandsantimano opened 2 months ago

rollandsantimano commented 2 months ago

Just came across your NSDI'20 paper, I was drawn to the workload analysis - thanks for the insights. Any pointers to tools / code used to collect & analyze queries ? I do see a couple of python notebooks for analysis of the dataset, and I was looking for tools used to prepare the dataset.

Here's what I'm trying to do: I'm a practitioner using, not building, different cloud-based OLTP & OLAP/DWH products. One top-of-mind concern is end-user cost attribution against usage of these products, especially when they are used by multiple workloads. The approach I've been pursuing is per-user quantitative & qualitative analysis of queries.

Yeah, writing a bunch of tools to create & analyze a query collection should be simple enough - but why re-invent the wheel ? :-)

Thanks in advance.