Discussion ticket.
The main goal of the data warehouse solution here is to enable querying data across different data sources (current tool used to do this is metabase).
The currently proposed architecture for the data warehouse is:
A single database that will house all clean growth data
Schemas for each data source within the central database
The warehouse data is read-only
An ETL that pulls data from the different sources (CIIP, SWRS, offsets?, CIF, OBPS) on a schedule
Potentially add some inital views that flatten and connect data for common workflows (will have to do some discovery with the business area on common workflows)
This ticket is to discuss the above approach and determine if this architecture makes sense for our use case, or if there are other ways to potentially design the warehouse.
Discussion ticket. The main goal of the data warehouse solution here is to enable querying data across different data sources (current tool used to do this is metabase). The currently proposed architecture for the data warehouse is:
This ticket is to discuss the above approach and determine if this architecture makes sense for our use case, or if there are other ways to potentially design the warehouse.