search

doujiang24 / lua-resty-kafka

Lua kafka client driver for the Openresty based on the cosocket API
BSD 3-Clause "New" or "Revised" License
801 stars 274 forks source link

flush_time，batch_size Settings don't work #79

Open caojingfu1990 opened 5 years ago

caojingfu1990 commented 5 years ago

Hello, everyone： local async_producer = producer:new(broker_list, {producer_type = "async" ,flush_time = 100000}) local ok, err = async_producer:send("access-log", product_id, message)

When I set the flush_time parameter, say 10 seconds. When I sent this message to kafka, did kafka have to wait 10 seconds to receive the message? Because batch_size defaults to 200, it should be sent to kafka when it is 10 seconds or when batch messages reach 200 messages. But I have set this parameter now, kafka can still receive the message immediately, is it that I understand the problem? I want to set these parameters to deal with high concurrency scenarios, thank you for your advice!

doujiang24 commented 5 years ago

@caojingfu1990 it sounds weird, you can try to add some debug log in the producer.lua, like: https://github.com/doujiang24/lua-resty-kafka/blob/master/lib/resty/kafka/producer.lua#L349