I have implement recently a simple shell script wrapper which uses kt and provides capabilities to use migrations (sort of) with Kafka, like automatic topics creation/deletion/reconfiguration, etc.
There is a common situation to create a lot of topics and during that Kafka may answer a little bit longer, which causes all subsequent operation to fail with the following error:
failed to create topic err=kafka server: Request exceeded the user-specified time limit in the request.
The default limit is 3sin Sarama, and it happens to be not enough quite often.
I've added the ability to configure this timeout via -timeout flag of kt admin command, or via KT_ADMIN_TIMEOUT env var globally.
fgeller
commented
5 years ago
I have implement recently a simple shell script wrapper which uses
ktand provides capabilities to use migrations (sort of) with Kafka, like automatic topics creation/deletion/reconfiguration, etc.There is a common situation to create a lot of topics and during that Kafka may answer a little bit longer, which causes all subsequent operation to fail with the following error:
The default limit is
3sin Sarama, and it happens to be not enough quite often.I've added the ability to configure this timeout via
-timeoutflag ofkt admincommand, or viaKT_ADMIN_TIMEOUTenv var globally.