confluentinc / kafka-connect-hdfs

Kafka Connect HDFS connector
Other
12 stars 396 forks source link

Add timeout for failed writes retry #663 #664

Open JozoVilcek opened 1 year ago

JozoVilcek commented 1 year ago

Problem

Retries on failed writes can go on for too long https://github.com/confluentinc/kafka-connect-hdfs/issues/663

Solution

Ability to specify a timeout for retries after which reads will reset from last committed offset.

Does this solution apply anywhere else?
If yes, where?

Test Strategy

Testing done:

Release Plan

JozoVilcek commented 1 year ago

Jenkins fails with error unrelated to my PR. I would appreciate hints on how to proceed

[ERROR] Failed to execute goal on project kafka-connect-hdfs: Could not resolve dependencies for 
project io.confluent:kafka-connect-hdfs:jar:10.3.0-SNAPSHOT: 
Failed to collect dependencies at io.confluent:kafka-connect-storage-hive:jar:11.1.4 -> org.apache.hive:hive-exec:jar:core:2.3.9 -> org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde: 
Failed to read artifact descriptor for org.pentaho:pentaho-aggdesigner-algorithm:jar:5.1.5-jhyde: 
Could not transfer artifact org.pentaho:pentaho-aggdesigner-algorithm:pom:5.1.5-jhyde 
from/to conjars (http://conjars.org/repo): 
transfer failed for http://conjars.org/repo/org/pentaho/pentaho-aggdesigner-algorithm/5.1.5-jhyde/pentaho-aggdesigner-algorithm-5.1.5-jhyde.pom: 
Connect to conjars.org:80 [conjars.org/54.235.127.59] failed: Connection timed out (Connection timed out) -> [Help 1]
cla-assistant[bot] commented 1 year ago

CLA assistant check
Thank you for your submission! We really appreciate it. Like many open source projects, we ask that you sign our Contributor License Agreement before we can accept your contribution.


Jozef Vilcek seems not to be a GitHub user. You need a GitHub account to be able to sign the CLA. If you have already a GitHub account, please add the email address used for this commit to your account.
You have signed the CLA already but the status is still pending? Let us recheck it.