It also has 2 behaviors: async and sync production. This behavior is triggered by the cfg.Sync config option.
In the async production case the errors occurring are passed over to the cfg.ProduceCallback if specified. This is correct.
In the sync production case errors occurring are not collected in any way and will never be returned to the caller. Worse: the caller will never know an error happened.
This way on cfg.Sync=true errors would be collected and returned. On cfg.Sync=false the code would not wait thus returning nil keeping the current behavior.
Produce()
method ofkafka.Producer
has this signature: https://github.com/elastic/apm-queue/blob/a508dbd8227928c843560343574741ffc4fca720/kafka/producer.go#L210It also has 2 behaviors: async and sync production. This behavior is triggered by the
cfg.Sync
config option.In the async production case the errors occurring are passed over to the
cfg.ProduceCallback
if specified. This is correct.In the sync production case errors occurring are not collected in any way and will never be returned to the caller. Worse: the caller will never know an error happened.
This is the relevant code where we produce to kafka: https://github.com/elastic/apm-queue/blob/a508dbd8227928c843560343574741ffc4fca720/kafka/producer.go#L243-L258
I'm not sure this behavior is intended and I would consider it a bug. The solution could be to collect errors with
errors.Join
and return the resulting error on line 261 https://github.com/elastic/apm-queue/blob/a508dbd8227928c843560343574741ffc4fca720/kafka/producer.go#L260-L264This way on
cfg.Sync=true
errors would be collected and returned. Oncfg.Sync=false
the code would not wait thus returningnil
keeping the current behavior.