search

datastax / starlight-for-kafka

DataStax - Starlight for Kafka
Apache License 2.0
17 stars 12 forks source link

When I set up token authentication for the pulsar cluster, the proxy cannot proxy kafka normally #27

Open tianshimoyi opened 1 year ago

tianshimoyi commented 1 year ago

Describe the bug When I set up token authentication for the pulsar cluster, the proxy cannot proxy kafka normally。

Version: Pulsar version: 2.9.3 Pulsar-kafka-proxy-2.10.0.6.nar pulsar-protocol-handler-kafka-2.9.3.18.nar (official version)

Broker config 截屏2022-12-05 下午5 42 43

Proxy config 截屏2022-12-05 下午5 42 51

Send message and result 截屏2022-12-05 下午5 43 00

tianshimoyi commented 1 year ago

@eolivelli can you help me ?

tianshimoyi commented 1 year ago

@eolivelli I modified the configuration file of the proxy and sent a message to report the following error

proxy.conf

截屏2022-12-06 下午6 37 46 截屏2022-12-06 下午6 36 26

log:

2022-12-06T18:33:04,422+0800 [pulsar-proxy-io-2-2] INFO  io.streamnative.pulsar.handlers.kop.KafkaProxyRequestHandler - handleRequestWithCoordinator MetadataRequest {topics=[public/default/catalina],allow_auto_topic_creation=true} system {}
2022-12-06T18:33:04,428+0800 [AsyncHttpClient-14-1] INFO  io.streamnative.pulsar.handlers.kop.KafkaProxyRequestHandler - found leader for persistent://public/__kafka/__consumer_offsets-partition-41: pulsar-kop-broker-2.pulsar-kop-broker.pulsar.svc.tcloud.gj:9092 (id: 1141621228 rack: null)
2022-12-06T18:33:04,428+0800 [AsyncHttpClient-14-1] INFO  io.streamnative.pulsar.handlers.kop.ConnectionToBroker - Opening proxy connection to pulsar-kop-broker-2.pulsar-kop-broker.pulsar.svc.tcloud.gj 9092 current user admin
2022-12-06T18:33:04,429+0800 [pulsar-proxy-io-2-2] INFO  io.streamnative.pulsar.handlers.kop.ConnectionToBroker - Connected to pulsar-kop-broker-2.pulsar-kop-broker.pulsar.svc.tcloud.gj:9092
2022-12-06T18:33:04,732+0800 [pulsar-external-web-5-6] INFO  org.eclipse.jetty.server.RequestLog - 10.0.20.174 - - [06/Dec/2022:18:33:04 +0800] "GET /metrics HTTP/1.1" 302 0 "-" "Prometheus/2.17.2" 1
2022-12-06T18:33:04,732+0800 [pulsar-proxy-io-2-2] INFO  io.streamnative.pulsar.handlers.kop.KafkaProxyRequestHandler - forgetMetadataForFailedBroker pulsar-kop-broker-2.pulsar-kop-broker.pulsar.svc.tcloud.gj:9092 -> [persistent://public/__kafka/__consumer_offsets-partition-41]
2022-12-06T18:33:04,733+0800 [pulsar-external-web-5-6] WARN  org.apache.pulsar.broker.web.AuthenticationFilter - [10.0.20.174] Failed to authenticate HTTP request: Authentication required
2022-12-06T18:33:04,733+0800 [pulsar-proxy-io-2-2] ERROR io.streamnative.pulsar.handlers.kop.ConnectionToBroker - Failed authentication against KOP broker UNKNOWN_SERVER_ERRORUser is not allowed to access this tenant
2022-12-06T18:33:04,733+0800 [pulsar-external-web-5-6] INFO  org.eclipse.jetty.server.RequestLog - 10.0.20.174 - - [06/Dec/2022:18:33:04 +0800] "GET /metrics/ HTTP/1.1" 401 540 "http://10.0.7.216:80/metrics" "Prometheus/2.17.2" 1
2022-12-06T18:33:04,733+0800 [pulsar-proxy-io-2-2] ERROR io.streamnative.pulsar.handlers.kop.ConnectionToBroker - error
java.util.concurrent.CompletionException: org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request.
        at io.streamnative.pulsar.handlers.kop.ConnectionToBroker.lambda$authenticateInternal$3(ConnectionToBroker.java:255) ~[?:?]
        at java.util.concurrent.CompletableFuture$UniApply.tryFire(CompletableFuture.java:642) ~[?:?]
        at java.util.concurrent.CompletableFuture.postComplete(CompletableFuture.java:506) [?:?]
        at java.util.concurrent.CompletableFuture.complete(CompletableFuture.java:2073) [?:?]
        at io.streamnative.pulsar.handlers.kop.ConnectionToBroker$ResponseFromBrokerHandler.channelRead(ConnectionToBroker.java:373) [UaTtTWGlGhu5-2e7szHtyQ/:?]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.fireChannelRead(ByteToMessageDecoder.java:327) [io.netty-netty-codec-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.handler.codec.ByteToMessageDecoder.channelRead(ByteToMessageDecoder.java:299) [io.netty-netty-codec-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.fireChannelRead(AbstractChannelHandlerContext.java:357) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.DefaultChannelPipeline$HeadContext.channelRead(DefaultChannelPipeline.java:1410) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:379) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.AbstractChannelHandlerContext.invokeChannelRead(AbstractChannelHandlerContext.java:365) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.DefaultChannelPipeline.fireChannelRead(DefaultChannelPipeline.java:919) [io.netty-netty-transport-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.epoll.AbstractEpollStreamChannel$EpollStreamUnsafe.epollInReady(AbstractEpollStreamChannel.java:800) [io.netty-netty-transport-classes-epoll-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.epoll.EpollEventLoop.processReady(EpollEventLoop.java:487) [io.netty-netty-transport-classes-epoll-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.channel.epoll.EpollEventLoop.run(EpollEventLoop.java:385) [io.netty-netty-transport-classes-epoll-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.util.concurrent.SingleThreadEventExecutor$4.run(SingleThreadEventExecutor.java:995) [io.netty-netty-common-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.util.internal.ThreadExecutorMap$2.run(ThreadExecutorMap.java:74) [io.netty-netty-common-4.1.77.Final.jar:4.1.77.Final]
        at io.netty.util.concurrent.FastThreadLocalRunnable.run(FastThreadLocalRunnable.java:30) [io.netty-netty-common-4.1.77.Final.jar:4.1.77.Final]
        at java.lang.Thread.run(Thread.java:829) [?:?]
Caused by: org.apache.kafka.common.errors.UnknownServerException: The server experienced an unexpected error when processing the request.
2022-12-06T18:33:04,734+0800 [pulsar-proxy-io-2-2] INFO  io.streamnative.pulsar.handlers.kop.KafkaProxyRequestHandler - forgetMetadataForFailedBroker pulsar-kop-broker-2.pulsar-kop-broker.pulsar.svc.tcloud.gj:9092 -> []