tulios / kafkajs

A modern Apache Kafka client for node.js
https://kafka.js.org
MIT License
3.76k stars 529 forks source link

Consumer "read_committed" isolation level #202

Closed ianwsperber closed 5 years ago

ianwsperber commented 6 years ago

To support transactions consumers should have an option to read only committed messages. In the Java API this is accomplished using the "isolation_level" option. The naive translation would be to provide an isolationLevel flag to our consumer() call.

Tasks

See the Java implementation for a reference on how to ignore aborted commits. Requires maintaining a map locally of aborted PIDs.

Along with #197 & #201 this should be all the work necessary for transactional consumers. @tulios LMK if this all sounds right to you.

ianwsperber commented 6 years ago

Our fetch implementation already defaults to the isolation level of read committed. This should continue to be the default behavior.

ianwsperber commented 6 years ago

Since we're going to default to "read_committed" behavior, we can hold off on providing a config option for isolation level until #210 (I'd mistakenly assumed we got "read_uncomitted" for free, but in practice the behavior we're getting is some weird mix of both)