cal-itp / data-infra

Cal-ITP data infrastructure
https://docs.calitp.org/data-infra
GNU Affero General Public License v3.0
48 stars 13 forks source link

Bring new NTD endpoints into the warehouse as marts #3472

Closed charlie-costanzo closed 1 week ago

charlie-costanzo commented 2 months ago

Description

Establish NTD mart tables based on new data pipeline.

Resolves #[issue]

How has this been tested?

local dbt

Post-merge follow-ups

Document any actions that must be taken post-merge to deploy or otherwise implement the changes in this PR (for example, running a full refresh of some incremental model in dbt). If these actions will take more than a few hours after the merge or if they will be completed by someone other than the PR author, please create a dedicated follow-up issue and link it here to track resolution.

github-actions[bot] commented 2 months ago

Warehouse report 📦

Checks/potential follow-ups

Checks indicate the following action items may be necessary.

New models 🌱

calitp_warehouse.mart.agency_service.dim_agencyservice\_2022__annual

calitp_warehouse.mart.agency_service.dim_agencyservice\_2022__average_typical_saturday

calitp_warehouse.mart.agency_service.dim_agencyservice\_2022__average_typical_sunday

calitp_warehouse.mart.agency_service.dim_agencyservice\_2022__average_typical_weekday

calitp_warehouse.mart.agency_service.dim_agencyservice\_2022__average_weekday_am_peak

calitp_warehouse.mart.agency_service.dim_agencyservice\_2022__average_weekday_midday

calitp_warehouse.mart.agency_service.dim_agencyservice\_2022__average_weekday_other

calitp_warehouse.mart.agency_service.dim_agencyservice\_2022__average_weekday_pm_peak

calitp_warehouse.mart.agency_service.dim_agencyservice\_2022__null_time_period

DAG

Legend (in order of precedence)

Resource type Indicator Resolution
Large table-materialized model Orange Make the model incremental
Large model without partitioning or clustering Orange Add partitioning and/or clustering
View with more than one child Yellow Materialize as a table or incremental
Incremental Light green
Table Green
View White

erikamov commented 2 months ago

I review the 2022 tables and I will be adding some missing columns soon from this PR

charlie-costanzo commented 1 week ago

this PR was supposed to be comprised of initial MVP tables made from assumptions based on early tickets, but has been made irrelevant by several more pointed PRs related to NTP mart tables that have been merged