camunda / connectors

Camunda Connectors
https://docs.camunda.io/docs/components/integration-framework/connectors/out-of-the-box-connectors/available-connectors-overview/
Apache License 2.0
35 stars 36 forks source link

Kafka inbound connector stops processing messages on any error #2750

Open chillleader opened 3 weeks ago

chillleader commented 3 weeks ago

Describe the Bug

The Kafka inbound connector stops processing messages in case of any error. To resume message processing, the user has to redeploy the process.

Because of this behavior, even if the user manually edits the offsets for the consumer group used by the connector, the connector won't be able to recover from this situation.

Steps to Reproduce

  1. Initialize an inbound Kafka connector
  2. Publish a message to the topic that contains invalid JSON, or simply doesn't match the activation condition specified in the connector
  3. Check the connector runtime logs, activity logs in the web modeler, and make sure that the consumer was stopped

Expected Behavior

The connector should retry after a certain backoff period (e.g. 30 seconds). The external consequences may lead to offsets changing, or the activation condition starting to evaluate to true, so the connector should be able to recover.

Environment