search

Datavault-UK / automate-dv

A free to use dbt package for creating and loading Data Vault 2.0 compliant Data Warehouses (powered by dbt, an open source data engineering tool, registered trademark of dbt Labs)
https://www.automate-dv.com
Apache License 2.0
513 stars 131 forks source link

[FEATURE] Add delete tracking for source records #187

Open sandervandewiel opened 1 year ago

sandervandewiel commented 1 year ago

**Is your feature request related to a problem? Yes, Our source system does hard-deletes. This does not seem to be reflected in the satellites.

Describe the solution you'd like Adding a deleted-indicator in the satellites and changing the loading to compare the data in the satellites to the source to see which records are deleted.

Describe alternatives you've considered I checked the code of effectivity satellites, but these are only meant to be used on links.

AB#5365

DVAlexHiggs commented 1 year ago

Hello. Thanks for this.

Record Tracking Satellites and Status Tracking Satellites, which are the Data Vault 2.0 standard approach to solving this issue are currently on our roadmap.

We would prefer to avoid diverging from standards and trying to implement them in the existing tables.

They are currently in ongoing development :)