search

adobe / experience-platform-streaming-connect

Experience platform Streaming Connectors for Data Ingress
Apache License 2.0
23 stars 20 forks source link

Integrate AEP sink connector with deadletter implementation provided by Kafka #36

Closed vivetiwa closed 2 years ago

vivetiwa commented 2 years ago

Integrate AEP sink connector with kafka sink connector dead letter support

Integrate AEP sink connector with dead letter feature supported by kafka replacing custom implementation of dead letter publish. Benefit of integration with kafka dead letter implementation :

  1. Same configuration used for kafka dead letter configuration could be used for AEP sink connector
  2. Commit offset could be handled by kafka connect itself based on error topic publish
  3. Kafka can handle management of kafka producer threads

Deadletter is available to sink connectors as part of :