search

MrPowers / mack

Delta Lake helper methods in PySpark
https://mrpowers.github.io/mack/
MIT License
286 stars 42 forks source link

Add type three scd upserts functionality #127

Open CommanderWahid opened 8 months ago

CommanderWahid commented 8 months ago

Add a new functionality to perform a type 3 scd upsert on a target delta table:

  1. Add type_3_scd_upsert function.
  2. Add unit test for type_3_scd_upsert function.
  3. Update README to include the new feature.
MrPowers commented 8 months ago

@CommanderWahid - thanks for submitting this PR.

I don't know much about Type 3 SCD, but I am trying to learn here.

I added some comments. Thanks for your patience here. I am going to have to learn as we go and we'll have to collaborate to make sure we get the right abstraction!