Move `kedro-catalog` JSON schema to `kedro-datasets`

kedro-org / kedro

Kedro is a toolbox for production-ready data science. It uses software engineering best practices to help you create data engineering and data science pipelines that are reproducible, maintainable, and modular.

https://kedro.org

Apache License 2.0

9.94k stars 903 forks source link

Move `kedro-catalog` JSON schema to `kedro-datasets` #4258

Open ianwhale opened 1 week ago

ianwhale commented 1 week ago

Description

Summary here is:

Datasets have moved, so it seems like this file should too.
Workflow for adding new datasets will be more seamless with the file living with kedro-datasets.

See more on the origin of this issue here: #4196.

Context

Alleviate maintenance burden by having these schema live with the datasets themselves.

Possible Implementation

Move it on over!
Add clause on "update json schema if necessary" to PR template.

Possible Alternatives

None considered!

datajoely commented 6 days ago

I would add that it might be time to explore automatic generation through introspection and avoid the manual maintenance element

ianwhale commented 6 days ago

If only Hacktober had more time! That would be a fun little project.

Great idea Joel!