search

streamnative / kop

Kafka-on-Pulsar - A protocol handler that brings native Kafka protocol to Apache Pulsar
https://streamnative.io/docs/kop
Apache License 2.0
450 stars 136 forks source link

[BUG] Cannot use Envoy 1.19.0 - invalid COMPACT_ARRAY length: 0 #665

Open eolivelli opened 3 years ago

eolivelli commented 3 years ago

Describe the bug The setup with the envoy proxy does not work, and I see these errors on the envoy proxy:

[2021-08-19 12:14:35.288][140117][debug][kafka] [source/extensions/filters/network/kafka/broker/filter.cc:112] could not process data from Kafka broker: invalid COMPACT_ARRAY length: 0

To Reproduce Install Pulsar 2.8.0, one zk, one bookie, one broker, with KOP (both current master and latest release of 2.8.0 branch). Run the broker, ensure that KOP works well.

I am on Mac, JDK8

change the kafkaAdvertisedListeners to kafkaAdvertisedListeners=PLAINTEXT://localhost:19092

so you have in broker.conf

brokerEntryMetadataInterceptors=org.apache.pulsar.common.intercept.AppendIndexMetadataInterceptor
kafkaListeners=PLAINTEXT://0.0.0.0:9092
kafkaAdvertisedListeners=PLAINTEXT://localhost:19092
messagingProtocols=kafka
protocolHandlerDirectory=./protocols
allowAutoTopicCreationType=partitioned

Start envoy proxy:

envoy -c configuration.yaml --log-level debug

configuration.yaml:

static_resources:
  listeners:
  - address:
      socket_address:
        # See KoP config item `kafkaAdvertisedListeners`
        address: 0.0.0.0
        port_value: 19092
    filter_chains:
    - filters:
      - name: envoy.filters.network.kafka_broker
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.kafka_broker.v3.KafkaBroker
          stat_prefix: kop-metrics
      - name: envoy.filters.network.tcp_proxy
        typed_config:
          "@type": type.googleapis.com/envoy.extensions.filters.network.tcp_proxy.v3.TcpProxy
          stat_prefix: tcp
          cluster: kop-cluster # It must be the same as the cluster name below
  clusters:
  - name: kop-cluster
    connect_timeout: 0.25s
    type: logical_dns
    lb_policy: round_robin
    load_assignment:
      cluster_name: localhost
      endpoints:
        - lb_endpoints:
          - endpoint:
              address:
                socket_address:
                  # See KoP config item `kafkaListeners`
                  address: 127.0.0.1
                  port_value: 9092

Start a console producer (tested with both Kafka 2.6.2 and 2.8.0 clients) bin/kafka-console-producer.sh --bootstrap-server localhost:19092 --topic test

Logs on the proxy

-08-19 12:18:22.660][145612][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:397] [C6] Creating connection to cluster kop-cluster
[2021-08-19 12:18:22.660][145612][debug][pool] [source/common/conn_pool/conn_pool_base.cc:245] trying to create new connection
[2021-08-19 12:18:22.660][145612][debug][pool] [source/common/conn_pool/conn_pool_base.cc:143] creating a new connection
[2021-08-19 12:18:22.660][145612][debug][connection] [source/common/network/connection_impl.cc:864] [C7] connecting to 127.0.0.1:9092
[2021-08-19 12:18:22.660][145612][debug][connection] [source/common/network/connection_impl.cc:880] [C7] connection in progress
[2021-08-19 12:18:22.660][145612][debug][conn_handler] [source/server/active_tcp_listener.cc:332] [C6] new connection from 127.0.0.1:56602
[2021-08-19 12:18:22.660][145612][debug][connection] [source/common/network/connection_impl.cc:669] [C7] connected
[2021-08-19 12:18:22.660][145612][debug][pool] [source/common/conn_pool/conn_pool_base.cc:289] [C7] attaching to next stream
[2021-08-19 12:18:22.660][145612][debug][pool] [source/common/conn_pool/conn_pool_base.cc:175] [C7] creating stream
[2021-08-19 12:18:22.660][145612][debug][filter] [source/common/tcp_proxy/tcp_proxy.cc:660] [C6] TCP:onUpstreamEvent(), requestedServerName: 
[2021-08-19 12:18:22.664][145612][debug][kafka] [source/extensions/filters/network/kafka/broker/filter.cc:112] could not process data from Kafka broker: invalid COMPACT_ARRAY length: 0
[2021-08-19 12:18:24.164][145612][debug][kafka] [source/extensions/filters/network/kafka/broker/filter.cc:112] could not process data from Kafka broker: no response metadata registered for correlation_id 2493
[2021-08-19 12:18:24.164][145612][debug][connection] [source/common/network/connection_impl.cc:637] [C7] remote close

Expected behavior The proxy works

eolivelli commented 3 years ago

my problem is solved if I use Kafka Clients 2.0.0.

it looks like KOP does not support APIVersion > 2, and then Envoy gets confused by the answer of the KOP broker