Docker compose initial start seems to be broken

[kg-emid]

I tried to set up application from scratch with docker compose (following this instruction) from 2 different machinse and for both of them rotor service always fails with message

docker-rotor-1      | 2023-11-22 15:08:51.852Z INFO  [singleton]: ️🚮🚮🚮 prisma disabled by setting DISABLE_SERVICE_prisma 
docker-rotor-1      | 2023-11-22 15:08:51.853Z INFO  [singleton]: ️🚮🚮🚮 pg disabled by setting DISABLE_SERVICE_pg 
docker-rotor-1      | 2023-11-22 15:08:51.869Z INFO  [singleton]: ️⚡️⚡️⚡️ redis connected in 2ms! 
docker-rotor-1      | 2023-11-22 15:08:51.931Z INFO  [singleton]: ️⚡️⚡️⚡️ redisLogger connected in 0ms! 
docker-rotor-1      | body-parser deprecated undefined extended: provide extended option node:internal/modules/cjs/loader:1241:14
docker-rotor-1      | 2023-11-22 15:08:51.934Z INFO  [rotor]: Starting kafka processing 
docker-rotor-1      | listening for signals
docker-rotor-1      | 2023-11-22 15:08:51.946Z INFO  [redis]: Redis @ redis://default:****@redis:6379 - successfully connected 
docker-rotor-1      | 2023-11-22 15:08:51.949Z INFO  [kafka-rotor]: Subscribing to kafka topics:  destination-messages,destination-messages-mt
docker-rotor-1      | {"level":"ERROR","timestamp":"2023-11-22T15:08:51.951Z","logger":"kafkajs","message":"[Connection] Response Metadata(key: 3, version: 6)","broker":"kafka:19093","clientId":"kafka-rotor","error":"This server does not host this topic-partition","correlationId":1,"size":602}
docker-rotor-1      | 2023-11-22 15:08:51.951Z ERROR [rotor]: Failed to start kafka processing 
docker-rotor-1      | │ KafkaJSProtocolError: This server does not host this topic-partition
docker-rotor-1      | │     at createErrorFromCode (/app/main.js:44384:10)
docker-rotor-1      | │     at Object.parse (/app/main.js:53027:11)
docker-rotor-1      | │     at Connection.send (/app/main.js:40316:35)
docker-rotor-1      | │     at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
docker-rotor-1      | │     at async [private:Broker:sendRequest] (/app/main.js:34363:14)
docker-rotor-1      | │     at async Broker.metadata (/app/main.js:33636:12)
docker-rotor-1      | │     at async /app/main.js:35128:25
docker-rotor-1      | │     at async /app/main.js:35561:14
docker-rotor-1      | │     at async Cluster.refreshMetadata (/app/main.js:35622:5)
docker-rotor-1      | │     at async Cluster.addMultipleTargetTopics (/app/main.js:35680:11)
docker-rotor-1      | 2023-11-22 15:08:51.952Z INFO  [kafka-rotor]: Closing kafka-rotor 
docker-rotor-1      | 2023-11-22 15:08:51.952Z INFO  [kafka-rotor]: Kafka-rotor closed gracefully. 💜 

After having closer look I figured out that destination-messages-mt topic in kafka wasn't create (but destination-messages was exist because it was probably created in bulker here). So for me it helps to create this topic manualy, but I think it should be either created automatically or mentioned somewhere in docs (auto-creation disabled for both consumer and producer)

[absorbb]

Hi @kg-emid ! Thank you for reporting that. Must be fixed in fresh rotor image

[fayas-akram]

I am still facing the issue. kafka failed to start due to unhealthy

[vklimontovich]

Hi @fayas-akram! We are working on improving self-hosted experience, so all the issues related to selg-hosted will be fixed altogether. See slack announcement: https://jitsuhq.slack.com/archives/C018G6W0URG/p1705002317138059