logs warning if deduplication state is large

dlt-hub / dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️

https://dlthub.com/docs

Apache License 2.0

2.65k stars 176 forks source link

Closed willi-mueller closed 3 days ago

willi-mueller commented 1 month ago

As a first step towards resolving this issue: warns if the deduplication state grows over a large count of hashes of the primary key.

netlify[bot] commented 1 month ago

Name	Link
Latest commit	ca3471cc58fcb45490699e91a198c4c998e8faec
Latest deploy log	https://app.netlify.com/sites/dlt-hub-docs/deploys/6733ba5613a0660008dca050