search

doujiang24 / lua-resty-kafka

Lua kafka client driver for the Openresty based on the cosocket API
BSD 3-Clause "New" or "Revised" License
801 stars 274 forks source link

feat: support kafka consumer #129

Closed bzp2010 closed 2 years ago

bzp2010 commented 2 years ago

Update

In this one PR, I have completed one of the simplest consumer implementations supporting two base APIs, ListOffsets and Fetch. This allows us to get messages from Kafka in the easiest way possible. However, ConsumerGroup and Kafka management offset are not supported yet, now offset can only be managed by the client itself.

But I haven't finished writing the test cases yet, you can review the code itself first, the tests will be done soon.

Usage

local c = consumer:new()
local ret, err = c:fetch()
local messages = ret.records
bzp2010 commented 2 years ago

Update

Test cases have been added, but for some reason it doesn't pass the tests in Travis CI, I presume because of the OpenResty version, maybe we can remove that CI now.

You are ready to start the REVIEW. @doujiang24

and cc @membphis

bzp2010 commented 2 years ago

@membphis The issues you pointed out have been fixed, and you can re-review them.

cc @doujiang24 Please help to check about deeper modifications, such as Kafka protocol related implementations.

doujiang24 commented 2 years ago

@bzp2010 great, I will take a deep look at this weekend.

bzp2010 commented 2 years ago

Update

The issues pointed out in the review have been modified and the documentation in the README has been added.

ping @doujiang24 @membphis Please check again.

doujiang24 commented 2 years ago

@bzp2010 Great job. Thanks!

bzp2010 commented 2 years ago

@bzp2010 Great job. Thanks!

@doujiang24 BTW, can you create a new release and publish it to luarocks? 🤔

doujiang24 commented 2 years ago

@bzp2010 yeah, a new release in my plan. I think it's better to have this PR in the new release. https://github.com/doujiang24/lua-resty-kafka/pull/128#discussion_r862444093