databricks / iceberg-kafka-connect

Apache License 2.0
219 stars 49 forks source link

Connector does not support error handling #191

Open mddunn opened 9 months ago

mddunn commented 9 months ago

The connector does not have support for error handling capabilities during CONVERT, TRANSFORM or PUT such as routing bad records to a dead letter queue

Example configurations that should be supported

errors.tolerance = all
errors.deadletterqueue.topic.name = { DLQ topic name }
errors.deadletterqueue.context.headers.enable = true
errors.log.enable = true

See the following blog for more details on connector error handling

yassan commented 9 months ago

Hi @mddunn ,

Regarding this issue, it has been brought up in Issue #152

I am not a maintainer of this repository, so I'm not privy to the detailed progress, but I believe the discussion in #152 could be informative and potentially helpful to you.