Discuss Data migration and ETL processes

feyclan / WT_warehouse

1 stars 0 forks source link

Discuss Data migration and ETL processes #10

Open croml opened 1 month ago

lucasschepers commented 1 month ago

To consider:

Identitfy per data source if we need to perform an "initial ETL" or an "Incremental ETL"
For the Incremental ETL, how how often do we require updates?

lucasschepers commented 1 month ago

Another consideration is how we will want to deal with slowly changing dimensions (SCD)'s. There are 3 basic ones, and we need to decide if we want type 1, 2 or 3.