Support for Exactly-once Semantic

[jsa4000]

Description

From Kafka v0.11 Exactly-once Semantics has already been implemented, however current dotnet version of Producer API doesn't allow this feature yet.

How to reproduce

Simply add "enable.idempotence" into the config

    var config = new Dictionary<string, object>()
        {
            {"bootstrap.servers", kafkaBrokerList.Trim() },
           { "enable.idempotence", true},
       };

        // Create the Producer
        Producer = new Producer<string, string>(config, new StringSerializer(Encoding.UTF8), new StringSerializer(Encoding.UTF8));

Checklist

Please provide the following information:

• [x] Confluent.Kafka nuget version: v0.11.2

• [x] Apache Kafka version: v0.11.0.1

• [x] Client configuration: bootstrap.servers + enable.idempotence

• [x] Operating system: Windows 10

• [x] Provide logs (with "debug" : "..." as necessary in configuration) "Parameter not Supported "enable.idempotence"

• [x] Provide broker log excerpts: None

• [x] Critical issue: False

[treziac]

This is a java configuration. Confluent-kafka use librdkafka, which don't implement idempotent client yet. You can track the issue here: https://github.com/edenhill/librdkafka/issues/1308

[jsa4000]

Thank you Treziac. I have already added +1 so we can have this feature in the next releases.

I have used this feature since the release of Kafka 0.11, but using the Java implementation as you said. For the rest of the platforms, I have just kept by using the old way. In this case, all messages have an unique key so they will be discarded at Producer side if repeated. As far as I know, and in general terms, that's basically what Kafka is doing in the background and sending into metadata. Also there are the Transactions that add lot of complexity from the develpment point of view. Sincerely, I though it was a lot easier to integrate the same functionality using other languages.

By the way, thanks for your excelent work.