TimeoutException on initTransaction version 5.1.2

[guntherrhon]

Hi, im using kafka instance from docker-compose (in attach my docker-compose.yml). Using no transactional mode i can connect, publish and correctly consume messages. If i use transactional way during initTransaction method i got TimeoutException:

• same client : org.apache.kafka.clients.*
• same client details the only differrences are the method to connect and to maintain the transaction(commit,abort etc)

================ MY YML FILE ================ version: '2.0' services:

kafka-topics-ui: ports:

• '8000:8000' environment:
• 'KAFKA_REST_PROXY_URL=http://MY_IP_HOST:8082'

• PROXY=true image: landoop/kafka-topics-ui

  zookeeper: image: confluentinc/cp-zookeeper:5.1.2 hostname: MY_IP_HOST ports:

  • "2181:2181"
  • "2888:2888"
  • "3888:3888" environment: ZOOKEEPER_CLIENT_PORT: 2181 ZOOKEEPER_TICK_TIME: 2000

  broker: image: confluentinc/cp-enterprise-kafka:5.1.2 hostname: MY_IP_HOST ports:

  • "9092:9092"
  • "29092:29092" environment: KAFKA_BROKER_ID: 1 KAFKA_ZOOKEEPER_CONNECT: 'MY_IP_HOST:2181' KAFKA_LISTENER_SECURITY_PROTOCOL_MAP: PLAINTEXT:PLAINTEXT,PLAINTEXT_HOST:PLAINTEXT KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://MY_IP_HOST:9092,PLAINTEXT_HOST://MY_IP_HOST:29092 LISTENERS: PLAINTEXT://0.0.0.0:9092 KAFKA_METRIC_REPORTERS: io.confluent.metrics.reporter.ConfluentMetricsReporter KAFKA_OFFSETS_TOPIC_REPLICATION_FACTOR: 1 KAFKA_GROUP_INITIAL_REBALANCE_DELAY_MS: 0 CONFLUENT_METRICS_REPORTER_BOOTSTRAP_SERVERS: MY_IP_HOST:9092 CONFLUENT_METRICS_REPORTER_ZOOKEEPER_CONNECT: MY_IP_HOST:2181 CONFLUENT_METRICS_REPORTER_TOPIC_REPLICAS: 1 CONFLUENT_METRICS_ENABLE: 'true' KAFKA_AUTO_CREATE_TOPICS_ENABLE: 'true' CONFLUENT_SUPPORT_CUSTOMER_ID: 'anonymous' KAFKA_CREATE_TOPICS: 'pds2.pagamento.bonificosepa.input:2:2,pds2.pagamento.bonificosepa.ok:2:2,pds2.pagamento.bonificosepa.ko:2:2,pds2.pagamento.bonificosepa.dlq:2:2,pds2.pagamento.bonificosepa.retry:2:2' schema-registry: image: confluentinc/cp-schema-registry:5.1.2 hostname: MY_IP_HOST ports:
  • "8081:8081" environment: SCHEMA_REGISTRY_HOST_NAME: MY_IP_HOST SCHEMA_REGISTRY_KAFKASTORE_CONNECTION_URL: 'MY_IP_HOST:2181'

  control-center: image: confluentinc/cp-enterprise-control-center:5.1.2 hostname: MY_IP_HOST ports:

  • "9021:9021" environment: CONTROL_CENTER_BOOTSTRAP_SERVERS: 'MY_IP_HOST:9092' CONTROL_CENTER_ZOOKEEPER_CONNECT: 'MY_IP_HOST:2181' CONTROL_CENTER_CONNECT_CLUSTER: 'connect:8083' CONTROL_CENTER_KSQL_URL: "http://MY_IP_HOST:8088" CONTROL_CENTER_KSQL_ADVERTISED_URL: "http://MY_IP_HOST:8088" CONTROL_CENTER_SCHEMA_REGISTRY_URL: "http://MY_IP_HOST:8081" CONTROL_CENTER_REPLICATION_FACTOR: 1 CONTROL_CENTER_INTERNAL_TOPICS_PARTITIONS: 1 CONTROL_CENTER_MONITORING_INTERCEPTOR_TOPIC_PARTITIONS: 1 CONFLUENT_METRICS_TOPIC_REPLICATION: 1 PORT: 9021

  ksql-server: image: confluentinc/cp-ksql-server:5.1.2 hostname: MY_IP_HOST ports:

  • "8088:8088" environment: KSQL_CONFIG_DIR: "/etc/ksql" KSQL_LOG4J_OPTS: "-Dlog4j.configuration=file:/etc/ksql/log4j-rolling.properties" KSQL_BOOTSTRAP_SERVERS: "MY_IP_HOST:9092" KSQL_HOST_NAME: ksql-server KSQL_APPLICATION_ID: "cp-all-in-one" KSQL_LISTENERS: "http://0.0.0.0:8088" KSQL_CACHE_MAX_BYTES_BUFFERING: 0 KSQL_KSQL_SCHEMA_REGISTRY_URL: "http://MY_IP_HOST:8081" KSQL_PRODUCER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringProducerInterceptor" KSQL_CONSUMER_INTERCEPTOR_CLASSES: "io.confluent.monitoring.clients.interceptor.MonitoringConsumerInterceptor"

  ksql-cli: image: confluentinc/cp-ksql-cli:5.1.2 entrypoint: /bin/sh tty: true

  rest-proxy: image: confluentinc/cp-kafka-rest:5.1.2 ports:

  • 8082:8082 hostname: MY_IP_HOST environment: KAFKA_REST_HOST_NAME: MY_IP_HOST KAFKA_REST_BOOTSTRAP_SERVERS: 'MY_IP_HOST:9092' KAFKA_REST_LISTENERS: "http://0.0.0.0:8082" KAFKA_REST_SCHEMA_REGISTRY_URL: 'http://MY_IP_HOST:8081'

    Where MY_IP_HOST is my linux ip vm. Thanks in advance Christian

[OneCricketeer]

1. hostname should not be set to the same name for each container
2. I'm surprised this works without network_mode: host on each service
3. The containers should talk to each other via the Docker network, not be routed through the host network. At the very least, start ove, and limit the problem... Start with just Kafka and Zookeeper, fixing these
   • KAFKA_ZOOKEEPER_CONNECT: 'zookeeper:2181'
   • KAFKA_ADVERTISED_LISTENERS: PLAINTEXT://kafka:9092,PLAINTEXT_HOST://MY_IP_HOST:29092

And test your transactional application.

note: KAFKA_CREATE_TOPICS does not work for these containers.