** Cannot get connection id for node 'VerneMQ@172.17.0.2'

[tuhin37]

I am running vernMQ container on server-1

docker run -it --name=vernmq1 -p 1883:1883 -p 8883:8883 -p 8080:8080 -p 44053:44053 -p 4369:4369 -p 8888:8888 -p 9100-9109:9100-9109 -e "DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on" -e "DOCKER_VERNEMQ_ACCEPT_EULA=yes" vernemq/vernemq Server-1 has IP: 192.168.1.109

I am creating a second docker container on server-2. And I want this one to join the container of server-1 and for a cluster.

docker run -it -p 1883:1883 -p 8883:8883 -p 8080:8080 -p 44053:44053 -p 4369:4369 -p 8888:8888 -p 9100-9109:9100-9109 -e "DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on" -e "DOCKER_VERNEMQ_ACCEPT_EULA=yes" -e "DOCKER_VERNEMQ_DISCOVERY_NODE=192.168.1.109" -e "DOCKER_VERNEMQ_DISCOVERY_NODE=192.168.1.109" --name vernmq2 vernemq/vernemq

I have mapped all exposed ports. The second container is not being able to join the first container on server-1

The interactive output of container-2

10:39:03.373 [info] loaded 0 subscriptions into vmq_reg_trie
10:39:03.380 [info] cluster event handler 'vmq_cluster' registered
10:39:04.161 [info] Sent join request to: 'VerneMQ@192.168.1.109'
10:39:11.166 [info] Unable to connect to 'VerneMQ@192.168.1.109'

The interactive output of container-1

10:38:06.791 [info] Opening LevelDB database at "./data/msgstore/12"
10:38:06.829 [info] Try to start vmq_generic_msg_store: ok
10:38:06.953 [info] loaded 0 subscriptions into vmq_reg_trie
10:38:06.962 [info] cluster event handler 'vmq_cluster' registered
10:39:04.180 [error] 
** Cannot get connection id for node 'VerneMQ@172.17.0.2'

Could anyone please help me with this. Thank you.

[tuhin37]

I tried creating a cluster with multiple containers on a single server. It worked beautifully. Also, I tested it with mqtt.fx (mqtt client) no problem there. The problem appears when I am running two containers on two different servers. Also, the servers are ubuntu-server 2020 virtualized on proxmox in my local network. I have checked the firewall and it is not active. As much I understood from the log is that the requesting container should pass a connection ID along with the request. If that is the case then is there an environment variable that I can set up with the ID?

[ioolkos]

@tuhin37 I don't think I have seen that specific error mentioning the connection id. You don't need to handle any connection id anyway. It'll be a variant of issues around network connectivity between those servers/pods. Noteably, you'll need to be able to reach the vmq listener directly on the configured IP (on the 44053 port, in case you use the default). This will not work exposing that port on a different IP, the original listener still only runs on the internal port.

So you need to ensure somehow that you have a Docker network connectivity on that level.

---

Please support the VerneMQ project: https://github.com/sponsors/vernemq

[tuhin37]

It worked when I used --network host in the docker run. Also, now I am running 3 container cluster and I am using a nginx layer4 load balancer.

events {}

stream {
    upstream mqtt_cluster {
        server 172.17.0.3:1883;
        server 172.17.0.4:1883;
        server 172.17.0.5:1883;
    }

    server {
        listen 1883;
        proxy_pass mqtt_cluster;
    }
}

The nginx LB is also running as a docker container on the same bridge. My mqtt client can only connect when all three containers are running. If I stop any one container, the mqtt client can not connect to the broker (Through LB)

Docker run commands

docker run -e "DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on" -e DOCKER_VERNEMQ_ACCEPT_EULA=yes --name mq1 -d vernemq/vernemq

docker run -e "DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on" -e DOCKER_VERNEMQ_ACCEPT_EULA=yes -e "DOCKER_VERNEMQ_DISCOVERY_NODE=172.17.0.3" --name mq2 -d vernemq/vernemq

docker run -e "DOCKER_VERNEMQ_ALLOW_ANONYMOUS=on" -e DOCKER_VERNEMQ_ACCEPT_EULA=yes -e "DOCKER_VERNEMQ_DISCOVERY_NODE=172.17.0.3" --name mq3 -d vernemq/vernemq

[ioolkos]

allow_register_during_netsplit = on
allow_publish_during_netsplit = on
allow_subscribe_during_netsplit = on
allow_unsubscribe_during_netsplit = on

---

Thank you for support the VerneMQ project: https://github.com/sponsors/vernemq

[tuhin37]

I am using the docker version, so should I pass these along as environment variables? If not environment variable then how? Also, I want the clients to connect using a self-signed TLS certificate and username and password. How to set it up?

Thank you