Use metering_history instead of warehouse_metering_history for consistency
Add in query acceleration costs
Update stg_metering_history and stg_warehouse_metering_history to re-process the last 7 days
In order to do this for stg_metering_history, we need to add a new field that is part of the unique key, entity_id. will require a full refresh.
Account for new WAREHOUSE_METERINGservice_type we are seeing in rate_sheet_daily
cost per query reconciliations:
check that everything lines up, ✅
with
existing as (
select
query_id,
start_time,
query_cost,
query_credits
from cost_per_query -- current model in prod
where start_time between '2023-11-01' and '2023-12-01'
),
new as (
select
query_id,
start_time,
query_cost,
query_credits
from dev.ian_dbt_snowflake_monitoring.cost_per_query
where start_time between '2023-11-01' and '2023-12-01'
)
select
*
from existing
full outer join new
on existing.query_id=new.query_id
where
existing.query_cost <> new.query_cost
or existing.query_credits <> new.query_credits
or existing.query_id is null
or new.query_id is null
;
check account with QA enabled, costs line up if you remove QA ✅
with
existing as (
select
query_id,
start_time,
query_cost,
query_credits
from cost_per_query -- prod model
where start_time between '2024-01-01' and '2024-01-05' -- spot checked multiple sets of dates
),
new as (
select
query_id,
start_time,
query_cost - query_acceleration_cost as query_cost,
query_credits - query_acceleration_credits as query_credits
from dev.ian_dbt_snowflake_monitoring.cost_per_query
where start_time between '2024-01-01' and '2024-01-05'
)
select
*
from existing
full outer join new
on existing.query_id=new.query_id
where
round(existing.query_cost, 4) <> round(new.query_cost, 4)
or round(existing.query_credits, 4) <> round(new.query_credits, 4)
or existing.query_id is null
or new.query_id is null
;
This PR does the following:
cost_per_query
model changesmetering_history
instead ofwarehouse_metering_history
for consistencystg_metering_history
andstg_warehouse_metering_history
to re-process the last 7 daysstg_metering_history
, we need to add a new field that is part of the unique key,entity_id
. will require a full refresh.WAREHOUSE_METERING
service_type
we are seeing inrate_sheet_daily
cost per query reconciliations:
check that everything lines up, ✅
check account with QA enabled, costs line up if you remove QA ✅