search

seanharr11 / etlalchemy

Extract, Transform, Load: Any SQL Database in 4 lines of Code.
MIT License
556 stars 83 forks source link

Upsert() should function on tables w/o a PK #24

Open Anmol-Tuple opened 7 years ago

Anmol-Tuple commented 7 years ago

Hi is there any method to remove only tables while upserting

--- Want to back this issue? **[Post a bounty on it!](https://www.bountysource.com/issues/46352895-upsert-should-function-on-tables-w-o-a-pk?utm_campaign=plugin&utm_content=tracker%2F41641218&utm_medium=issues&utm_source=github)** We accept bounties via [Bountysource](https://www.bountysource.com/?utm_campaign=plugin&utm_content=tracker%2F41641218&utm_medium=issues&utm_source=github).
seanharr11 commented 7 years ago

Could you be a more specific? I'm not sure exactly what you mean...

Anmol-Tuple commented 7 years ago

Hi , when I am making drop_database=false each time the data is appending to the table thus making duplicates records

seanharr11 commented 7 years ago

Does the table with the duplicate records have a primary key? I believe we use the primary key of the table to determine whether a row should be updated or inserted.

nitishxp commented 7 years ago

what if it does not contain primary keys?

seanharr11 commented 7 years ago

There should be 2 fallback options if no primary keys are present on the table (in order):

  1. We gather all FKs, and use them as a "composite primary key" to upsert() on. This will be a common case when dealing with association tables (in many-to-many relations).
  2. We specify the columns that make up the "composite primary key", as a part of the config, for each table that does not have a primary key.