The connector is committing offsets to Kafka even if it is not able to send data to AEP for example due to authentication or network errors. This may cause data loss in streaming process.
Steps to reproduce
Starts the connector with a wrong authentication configuration
Send some data to Kafka
Monitor consumer group lag
Expected behaviour
The expected behaviour is that in case of error lag will increase and as soon as the problem as been solved, the connector will start to consume all the unprocessed messages.
Actual behaviour
The actual behavious is that the lag reach 0 and the messages are not processed after the problem is solved and so they are lost.
vinodrajmh
commented
3 years ago
Subject of the issue
The connector is committing offsets to Kafka even if it is not able to send data to AEP for example due to authentication or network errors. This may cause data loss in streaming process.
Steps to reproduce
Expected behaviour
The expected behaviour is that in case of error lag will increase and as soon as the problem as been solved, the connector will start to consume all the unprocessed messages.
Actual behaviour
The actual behavious is that the lag reach 0 and the messages are not processed after the problem is solved and so they are lost.