ConduitIO / conduit

Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
https://conduit.io
Apache License 2.0
363 stars 42 forks source link

Schema support #1560

Open hariso opened 1 month ago

hariso commented 1 month ago

Feature description

Adds support for schemas in Conduit. The work is split as in the lists below. It would be good if we start with #1561 and the conduit-commons issues (minus schema auto-generation) so we can parallelize the rest of the work.

Conduit

conduit-commons

Connector SDK

Processor SDK

lovromazgon commented 1 month ago

A lot, if not all of the code, needed for auto-generating Avro schemas can be found here.

The utilities mentioned in both SDKs can be placed in conduit-commons, so we can re-use it.