streamingfast / substreams-sink-sql

Apache License 2.0
12 stars 15 forks source link

Error when inserting a table with unique constraints #51

Open sdevkc opened 11 months ago

sdevkc commented 11 months ago

When inserting data with unique constraints it throws an error.

For example when slot and program_id is not unique:

CREATE TABLE IF NOT EXISTS trans (
    "signature" VARCHAR(256) PRIMARY KEY,
    "slot" VARCHAR(256),
    "program_id" VARCHAR(256),
    CONSTRAINT uq_tran UNIQUE ("slot", "program_id")
);

Is there an option for handling conflicts? E.g.,

INSERT INTO ... ON CONFLICT DO NOTHING;
maoueh commented 11 months ago

There is no current built-in support for conflicting elements. Doesn't it simply mean you are trying to insert twice the same (slot,program_id)?

sdevkc commented 11 months ago

Yes, for my table there are other columns but some table might change while the indexer is running so conflicts could happen.

ptescher commented 9 months ago

We are in a similar boat, it would be nice to have an upsert option for rows. On conflict set ... for instance.

abourget commented 9 months ago

How would you express that in the DatabaseChanges model? Specify the things to set, the operation to run, on conflict? With a set-only approach, we can do high speed inserts through COPY FROM, if we start enabling operations,, hmmm . Might change the possibilities