Post "kafka:///": unsupported protocol scheme "kafka"

Environment info:

KrakenD version: Version: 2.0.6
System info: docker-compose
Backend technology: devopsfaith/krakend bitnami/kafka:latest

Describe the bug

For some reason, krakend can't post to kafka

krakend_1    | 2022/08/30 16:02:44 KRAKEND ERROR: [ENDPOINT: /v1/data] Post "kafka:///": unsupported protocol scheme "kafka"
krakend_1    | [GIN] 2022/08/30 - 16:02:44 | 500 |     382.309µs |      172.25.0.1 | POST     "/v1/data"

Your configuration file:

Please checkout my environment: https://github.com/Vido/krakend-kafka-poc

This repo has all you need to reproduce this error

I used different backend version, but I always get the same:

github.com/devopsfaith/krakend-pubsub/publisher
github.com/krakendio/krakend-pubsub/publisher
backend/pubsub/publisher

{
  "$schema": "https://www.krakend.io/schema/v3.json",
  "version": 3,
  "name": "KrakenD - API Gateway",
  "timeout": "30s",
  "cache_ttl": "300s",
  "endpoints": [
    {
      "endpoint": "/v1/data",
      "method": "POST",
      "output_encoding": "json",
      "backend": [
        {
          "host": ["kafka://"],
          "disable_host_sanitize": true,
          "extra_config": {
            "backend/pubsub/publisher": {
              "topic_url": "my_favorite_topic"
            }
          }
        }
      ],
      "input_headers": [
        "Host"
      ]
    }
  ]
}

Commands used Please, check the repo link above

# Setup the env
docker-compose up

# make requests
python krakend_client.py

Expected behavior A clear and concise description of what you expected to happen.

Logs

krakend_1    | 2022/08/30 16:02:44 KRAKEND ERROR: [ENDPOINT: /v1/data] Post "kafka:///": unsupported protocol scheme "kafka"
krakend_1    | [GIN] 2022/08/30 - 16:02:44 | 500 |     382.309µs |      172.25.0.1 | POST     "/v1/data"

I found out the problem:

Kafka was not ready to accept connections.

Krankend tries to create pubsub, it can't connect, it fails to create

2022/09/02 04:21:52 KRAKEND ERROR: [BACKEND: kafka://my_favorite_topic][PubSub]%!(EXTRA string=kafka: client has run out of available brokers to talk to: dial tcp [172.28.0.3:9093](http://172.28.0.3:9093/): connect: connection refused)

When the request comes, it cant find the pubsub

2022/08/30 16:02:44 KRAKEND ERROR: [ENDPOINT: /v1/data] Post "kafka:///": unsupported protocol scheme "kafka"

Solution: add some heathcheck to kafka service

  kafka:
    image: 'bitnami/kafka:latest'
    healthcheck:
      test: ["CMD-SHELL", "kafka-topics.sh --bootstrap-server 127.0.0.1:9092 --topic my_favorite_topic --describe"]
      interval: 5s
      timeout: 5s
      retries: 5

  krakend:
    image: devopsfaith/krakend:2.0.6
      - "8080:8080"
    depends_on:
      kafka:
        condition: service_healthy

krakend / krakend-pubsub

Post "kafka:///": unsupported protocol scheme "kafka" #17