search

confluentinc / librdkafka

The Apache Kafka C/C++ library
Other
278 stars 3.15k forks source link

Error handling question "treat any error that has neither retriable, abortable, or fatal set, as a fatal error" #4554

Open flojdek opened 11 months ago

flojdek commented 11 months ago

Reading the documentation, specifically this part: https://github.com/confluentinc/librdkafka/blob/master/INTRODUCTION.md#error-handling-1 I'm a bit confused about the meaning of what is meant by "set" in that part of the documentation:

For all other errors returned from the transactional API: the current recommendation is to treat any error that has neither retriable, abortable, or fatal set, as a fatal error.

Looking at the code there's no option/optional/nullable return type being used to convey to the client the information whether the error is retriable and/or abortable and/or fatal. I can see the return value is boolean which is not entirely suitable for communicating that there are 3 options here: not set, true, false.

Is the above documentation to be understood that if all three properties of the error are false then it's the same as if the library communicates to us that these are actually not set and all bets are off how to handle those errors and it is a recommendation to the application to treat those as fatal.

I'd imagine that is done that way as to not break previous clients?

nitinsharma-sgp commented 11 months ago

this is a interface design issue ... and am confused how to handle this in our application. How can I request for interface upgrade for this functionality and get this prioritized?

emasab commented 8 months ago

'Set' here means that they are true, not that they can be set or unset. So if all of rd_kafka_error_is_retriable, rd_kafka_error_is_fatal and rd_kafka_error_txn_requires_abort are false, then the reccomendation is to treat it as a fatal error. So to recreate the producer and possibly the consumer(s) whose offsets are included in the transaction too.

flojdek commented 8 months ago

Thanks @emasab for clarification, so as I expected. 'Set' has the meaning of kind of 'bit set' which means true. I would still update the docs to be clearer here.

emasab commented 7 months ago

@flojdek Would this be clearer? For all other errors returned from the transactional API: the current recommendation is to treat any error that has neither retriable, abortable, or fatal functions returning true, as a fatal error.