Galileo-Galilei / kedro-pandera

A kedro plugin to use pandera in your kedro projects
https://kedro-pandera.readthedocs.io/en/latest/
Apache License 2.0
33 stars 5 forks source link

Generate metadata catalog entry from annotated functions #26

Open Galileo-Galilei opened 1 year ago

Galileo-Galilei commented 1 year ago

Description

TBD

Context

TBD

Possible Implementation

TBD

Possible Alternatives

TBD

datajoely commented 1 year ago

So in this comment Neils the maintainer makes the point that

"validation logic needs to be decoupled from more opinionated storage systems, so being able to write your constraint logic in plain Python allows for portability of your data validation layer (i.e. you don't get stuck with a particular set of opinions at the storage layer)"

which I think is the tension we're feeling between inline checks and declarative catalog entries.

https://www.linkedin.com/posts/nbantilan_python-activity-7099398333546315776--Qay?utm_source=share&utm_medium=member_desktop

Galileo-Galilei commented 1 year ago

Ah ah I saw the same post and had the exact same feeling.

datajoely commented 1 year ago

I think we should support both, but push users towards the pythonic API the more I think about it