Unhandled Confluent.Kafka.KafkaException when attempting to produce with ACLs in place

[htims1989]

Description

Attempting to produce with ACLs in place that deny access causes producer.ProduceAsync to throw:

An unhandled exception of type 'Confluent.Kafka.KafkaException' occurred in Confluent.Kafka.dll Additional information: Local: Unknown topic

consumer.Poll correctly returns an error via the consumer.OnError event:

Error consuming from topic/partition/offset /0/0: Broker: Group authorization failed

How to reproduce

Enable client authentication on the broker and set up an ACL using the certificates Subject to deny access. Attempt to produce using producer.ProduceAsync.

Checklist

Please provide the following information:

• [x] Confluent.Kafka nuget version: 0.9.5
• [x] Apache Kafka version: 0.11.0.0
• [x] Client configuration: { "bootstrap.servers", brokerList }, { "api.version.request", "true" }, { "security.protocol", "SSL" }, { "ssl.ca.location", @"C:\Users\luke smith\Desktop\gen\ca-cert" }, { "ssl.certificate.location", @"C:\Users\luke smith\Desktop\gen\kafkaadminclient.pem" }, { "ssl.key.location", @"C:\Users\luke smith\Desktop\gen\kafkaadminclient.key" }, { "ssl.key.password", "abcdefgh" }
• [x] Operating system: Broker: Ubuntu 16.04 Client: Windows 10
• [ ] Provide logs (with "debug" : "..." as necessary in configuration) Unsure what to provide.
• [ ] Provide broker log excerpts Nothing logged to the broker logs.
• [ ] Critical issue I guess a good way to tell a client to go away is to crash it :P

[treziac]

https://github.com/confluentinc/confluent-kafka-dotnet/wiki/Producing-messages#important-notes

At current time, ProduceAsync may throw exception in some case (message too big for example). This may change in a future release (no throw but returns a message with Error instead)

To be more clear, on first send, ProduceAsync will success but DeliveryHandler will contain an error (unknown topic) Next call will directly return by throwing without trying to send the data because the error is considered permanent (need manual intervention) It will try again if metadata which are called frequently show the topic has been created

You have to call ProduceAsync in a Try/Catch + check the message.Error to know if an error occured. This is tracked in #203

[mhowlett]

tracking in #203