confluentinc / kafka-connect-bigquery

A Kafka Connect BigQuery sink connector
Apache License 2.0
3 stars 2 forks source link

Storage Write API Bug Fix: Task failure due to concurrent commit #331

Closed b-goyal closed 1 year ago

b-goyal commented 1 year ago

commitIfEliglble methods gets called by scheduler as well as on completion of append calls. There is a race condition which can lead to 'STREAM ALREADY COMMITTED' error. This PR fixes task failure due to concurrent stream commit by adding synchronisation and adding conditional check before attempting commit. This PR adds more logging .