Running SOCKS5 Proxy in docker-compose

[mario-s]

Hi,

I have a question, regarding a SOCKS5 proxy within the same network of a Kafka cluster. All Kafka nodes and the Zookeeper are defined in a Docker compose file. They can interact with each other. But the moment I add a proxy and try to get access to it via the proxy, nothing happens. Is there something wrong with the setup for the proxy? I'm really not sure if the mapping is correct.

version: "3"

services:
  zookeeper:
    image: docker.io/bitnami/zookeeper:3.7
    ports:
      - "2181:2181"
    environment:
      - ALLOW_ANONYMOUS_LOGIN=yes
    volumes:
      - zookeeper_data:/bitnami/zookeeper
  kafka-0:
    image: docker.io/bitnami/kafka:3
    ports:
      - "9093:9093"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_CFG_BROKER_ID=0
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
      - KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://:9093
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka-0:9092,EXTERNAL://localhost:9093
      - KAFKA_INTER_BROKER_LISTENER_NAME=CLIENT
    volumes:
      - kafka_0_data:/bitnami/kafka
    depends_on:
      - zookeeper
  kafka-1:
    image: docker.io/bitnami/kafka:3
    ports:
      - "9094:9094"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_CFG_BROKER_ID=1
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
      - KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://:9094
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka-1:9092,EXTERNAL://localhost:9094
      - KAFKA_INTER_BROKER_LISTENER_NAME=CLIENT
    volumes:
      - kafka_1_data:/bitnami/kafka
    depends_on:
      - zookeeper
  kafka-2:
    image: docker.io/bitnami/kafka:3
    ports:
      - "9095:9095"
    environment:
      - KAFKA_CFG_ZOOKEEPER_CONNECT=zookeeper:2181
      - KAFKA_CFG_BROKER_ID=2
      - ALLOW_PLAINTEXT_LISTENER=yes
      - KAFKA_CFG_LISTENER_SECURITY_PROTOCOL_MAP=CLIENT:PLAINTEXT,EXTERNAL:PLAINTEXT
      - KAFKA_CFG_LISTENERS=CLIENT://:9092,EXTERNAL://:9095
      - KAFKA_CFG_ADVERTISED_LISTENERS=CLIENT://kafka-2:9092,EXTERNAL://localhost:9095
      - KAFKA_INTER_BROKER_LISTENER_NAME=CLIENT
    volumes:
      - kafka_2_data:/bitnami/kafka
    depends_on:
      - zookeeper
  proxy-tools:
    image: grepplabs/kafka-proxy:v0.3.1-all
    ports:
      - "1080:1080"
    command:
      - 'tools'
      - 'socks5-proxy'
      - '--addr'
      - 'localhost:1080'
  proxy-server:
    image: grepplabs/kafka-proxy:v0.3.1-all
    command:
      - 'server'
      - '--bootstrap-server-mapping=kafka-0:9092,127.0.0.1:32400'
      - '--bootstrap-server-mapping=kafka-1:9092,127.0.0.1:32401'
      - '--bootstrap-server-mapping=kafka-2:9092,127.0.0.1:32402'
      - '--forward-proxy'
      - 'socks5://localhost:1080'

volumes:
  zookeeper_data:
    driver: local
  kafka_0_data:
    driver: local
  kafka_1_data:
    driver: local
  kafka_2_data:
    driver: local

[everesio]

Please modify you docker-compose.yaml

  proxy-tools:
    image: grepplabs/kafka-proxy:v0.3.1-all
    ports:
      - "1080:1080"
    command:
      - 'tools'
      - 'socks5-proxy'
      - '--addr'
      - '0.0.0.0:1080'
  proxy-server:
    image: grepplabs/kafka-proxy:v0.3.1-all
    ports:
      - "32400:32400"
      - "32401:32401"
      - "32402:32402"
    command:
      - 'server'
      - '--bootstrap-server-mapping=kafka-0:9092,0.0.0.0:32400'
      - '--bootstrap-server-mapping=kafka-1:9092,0.0.0.0:32401'
      - '--bootstrap-server-mapping=kafka-2:9092,0.0.0.0:32402'
      - '--forward-proxy=socks5://proxy-tools:1080'
      - '--debug-enable'

and from the host try out the clients

kafka-console-producer.sh --broker-list 127.0.0.1:32400 --topic test
kafka-console-consumer.sh --bootstrap-server 127.0.0.1:32401 --topic test

Tested on the linux desktop.

[mario-s]

Yes, this works for me as well. Thanks a lot!