cagov / data-infrastructure

CalData infrastructure
https://cagov.github.io/data-infrastructure
MIT License
7 stars 0 forks source link

Create data models for Snowflake cost tracking #427

Open ian-r-rose opened 23 hours ago

ian-r-rose commented 23 hours ago

We are currently not doing a good job of tracking our Snowflake credit usage. We should create some data models here to give us more visibility into costs.

Most of the relevant information will be in the SNOWFLAKE.ORGANIZATION_USAGE schema. Some views (non-exhaustive) that track credit usage are:

  1. AUTOMATIC_CLUSTERING_HISTORY
  2. DATA_TRANSFER_HISTORY
  3. DATABASE_STORAGE_USAGE_HISTORY
  4. MATERIALIZED_VIEW_REFRESH_HISTORY
  5. METERING_DAILY_HISTORY
  6. PIPE_USAGE_HISTORY
  7. STAGE_STORAGE_USAGE_HISTORY
  8. STORAGE_DAILY_HISTORY
  9. WAREHOUSE_METERING_HISTORY

There are also a couple of views that would be helpful if we could use them, but do not work when Snowflake is purchased through a reseller:

  1. USAGE_IN_CURRENCY_DAILY
  2. REMAINING_BALANCE_DAILY