help request: Apisix KafKa Integration

apache / apisix

The Cloud-Native API Gateway

https://apisix.apache.org/blog/

Apache License 2.0

13.97k stars 2.45k forks source link

help request: Apisix KafKa Integration #10925

Open senolatac opened 5 months ago

senolatac commented 5 months ago

Description

I saw there is kafka plugin in Apisix and there is a documentation https://apisix.apache.org/docs/apisix/next/pubsub/kafka/ but I couldn't understand how I use the apisix route after creation. I though it is a broker url but it is not.

Any suggestion?

Environment

APISIX version (run apisix version): 3.2.0
Operating system (run uname -a):
OpenResty / Nginx version (run openresty -V or nginx -V):
etcd version, if relevant (run curl http://127.0.0.1:9090/v1/server_info):
APISIX Dashboard version, if relevant:
Plugin runner version, for issues related to plugin runners:
LuaRocks version, for installation issues (run luarocks --version):

shreemaan-abhishek commented 5 months ago

I though it is a broker url but it is not.

what is a broker URL? Please be specific about your issues.

shreemaan-abhishek commented 5 months ago

The kafka-logger: https://docs.api7.ai/hub/kafka-logger plugin is used to log entries to kafka and the pubsub documentation link you shared is for proxying kafka.

kayx23 commented 4 months ago

Disclaimer: I'm no expert on this topic, but here are some information I gathered.

The test case (https://github.com/apache/apisix/blob/master/t/pubsub/kafka.t#L156) suggests that one should pull messages from kafka over websocket connections and with messages binary-encoded.

Only list_offset and fetch commands are supported at the moment. list_offset is used to check the message offset at the specified timestamp on the current topic. fetch is used to obtain the message from a specific offset.