dlt-hub / dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
https://dlthub.com/docs
Apache License 2.0
2.38k stars 154 forks source link

Default hint of "unique" does not add a unique index #1848

Open boxydog opened 1 week ago

boxydog commented 1 week ago

dlt version

1.0.0

Describe the problem

Default hint of "unique" does not add a unique index.

See my test code and the Postgres table description below. id2 has a unique index (because I explicitly declared one), but id does not.

                   Table "public.a_test_unique"
    Column    |       Type        | Collation | Nullable | Default 
--------------+-------------------+-----------+----------+---------
 id           | bigint            |           |          | 
 id2          | bigint            |           |          | 
 _dlt_load_id | character varying |           | not null | 
 _dlt_id      | character varying |           | not null | 
Indexes:
    "a_test_unique__dlt_id_key" UNIQUE CONSTRAINT, btree (_dlt_id)
    "a_test_unique_id2_key" UNIQUE CONSTRAINT, btree (id2)

Expected behavior

I declare a default hint that "id" should be unique. For the Postgres destination, "unique" should put a unique index on the field. It does not.

Steps to reproduce

id should have a unique index, but it does not. Separately, id2 does have a unique index, but it was declared explicitly.

test_unique_hint.tgz

Operating system

macOS

Runtime environment

Local

Python version

3.11

dlt data source

json

dlt destination

Postgres

Other deployment details

No response

Additional information

Using hints to declare indexes may not be supported behavior. It's not documented anywhere.

However, it is what I expected, given https://dlthub.com/docs/devel/dlt-ecosystem/destinations/postgres#supported-column-hints says a unique field will get an index.