MrPowers / mack

Delta Lake helper methods in PySpark
https://mrpowers.github.io/mack/
MIT License
286 stars 42 forks source link

Brainstorm a generic appender interface #107

Open MrPowers opened 1 year ago

MrPowers commented 1 year ago

We have two different types of appends now in mack:

We might want to think of a more generic interface that will allows for mixing & matching of the different append modes. Users might want to use both.

Perhaps something like this: mack.appender.without_duplicates(delta_table).constrained(delta_table, quarantine_table).

Are we going to want to add other types of append functionality as well?