search

dlt-hub / dlt

data load tool (dlt) is an open source Python library that makes data loading easy 🛠️
https://dlthub.com/docs
Apache License 2.0
2.65k stars 176 forks source link

[PoC] - prevent leaking of secrets when running validate_dict #2027

Open sh-rp opened 1 week ago

sh-rp commented 1 week ago

Description

This change will require the DictValidationException to have the full incoming validatable object. We extract all strings from this object and make sure that the error message does not have any of these in clear text.

We could probably allow the user to set an env var to disable this obfuscator for development.

Originally the plan was to also do stuff to prevent leakage from the configs, but the __str__ method of configs currently is not implemented and just returns the default python object identifier. So I don't think there is anything to be done there.

netlify[bot] commented 1 week ago

Deploy Preview for dlt-hub-docs canceled.

Name Link
Latest commit 840832c6faa30793466c7029bdd98c72157b4f8c
Latest deploy log https://app.netlify.com/sites/dlt-hub-docs/deploys/672a52b15f37540008342ae1
sh-rp commented 1 week ago

@willi-mueller what where the arguments against implementing it this way?