linkedin / li-apache-kafka-clients

li-apache-kafka-clients is a wrapper library for the Apache Kafka vanilla clients. It provides additional features such as large message support and auditing to the Java producer and consumer in the open source Apache Kafka.
BSD 2-Clause "Simplified" License
133 stars 52 forks source link

Fix race condition in LiKafkaInstrumentedProducerImpl #189

Closed kehuum closed 4 years ago

kehuum commented 4 years ago

Fix race condition in LiKafkaInstrumentedProducerImpl when two threads calling close in send callback.

Problem: BMM has a use case of calling producer.close in send callback when send fails, with current code, there could still be race conditions given the following sequence:

Both Thread 1 and Thread 2 are calling same producer.send(tp, cbk) and fails, and tries to call producer.close in the cbk:

Thread 1: Hold closeLock, (release Read lock,) wait on Write lock Thread 2: Hold Read lock, wait on closeLock.

Thus deadlock.

Fix: Ensure same ordering of acquiring/releasing locks when proceedClose(), i.e. first release all Read locks held by current thread, then grab Write lock to proceed closing.