DataWorkz-NL / KubeETL

ETL controller for Kubernetes
Apache License 2.0
4 stars 0 forks source link

Feat/add DataSet kind #10

Closed Blokje5 closed 3 years ago

Blokje5 commented 3 years ago

The DataSource kind is another abstraction that utilised the Connection. The DataSource allows users to define additional metadata to fetch the DataSource or send data to the DataSource. It defines a schema field which allows users to define a Connection from which to fetch the schema for the DataSource. Additionally it has a metadata field for defining additional properties, e.g. the table name of where the data resides.

As with the Connection, there is a DataSourceType to validate the DataSource dynamically. This allows users to define the structure of the DataSource.

ThijsKoot commented 3 years ago

was lgtm'd :+1: