`kafka-connect` Build a Kafka Source connector to pull Adobe IO Events and stream these in Kafka

adobe / aio-lib-java

Adobe I/O - Java SDK

https://opensource.adobe.com/aio-lib-java/

Apache License 2.0

5 stars 18 forks source link

`kafka-connect` Build a Kafka Source connector to pull Adobe IO Events and stream these in Kafka #159

Open francoisledroff opened 1 year ago

francoisledroff commented 1 year ago

Build a Kafka Source kafka-connect connector :

that leverages aio-lib-java sdk to pull Adobe IO Events from the journaling API
and stream these Events in Kafka.

see https://kafka.apache.org/documentation/#connect

OneCricketeer commented 1 year ago

There already exists a sink connector. Perhaps add the source connector class to that?

One concern I had with the journalling API, however was that it presents a "single partitioned" endpoint, so something to either consider for Kafka consumer scalability, or to document that connector producer should write to a single partitioned topic.