localhost mqtt broker with ditto connection

[Gauravmeenu]

Actually, I'm using MQTT broker in localhost with the help of docker image, and along with I'm also running eclipse ditto with docker-compose up as given in the ditto deployment GitHub repo. whenever I'm trying to testconnection with mqtt tcp localhost protocol it gives me an error like as given below .

{ "status": 504, "error": "connectivity:connection.failed", "message": "The Connection with ID 'mqtt-connection-source44' failed to connect.", "description": "Could not establish a connection on 'localhost:1883'. Make sure the endpoint is reachable and that no firewall prevents the connection." }

But when I'm trying to connect with mosquitto public server test.mosquitto.org it works fine.

testconnection file given below

{ "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:testConnection", "connection": { "id": "mqtt-connection-source44", "connectionType": "mqtt", "connectionStatus": "open", "failoverEnabled": true, "uri": "tcp://localhost:1883", "sources": [{ "addresses": ["my.sensors/#"], "authorizationContext": ["nginx:ditto"], "qos": 0, "filters": [] }], "mappingContext": { "mappingEngine": "JavaScript", "options": { "incomingScript": "function mapToDittoProtocolMsg(headers, textPayload, bytePayload, contentType) {const jsonString = String.fromCharCode.apply(null, new Uint8Array(bytePayload)); const jsonData = JSON.parse(jsonString); const thingId = jsonData.thingId.split(':'); const value = {measurements: {properties: {temperature: jsonData.temperature, humidity: jsonData.humidity}}}; return Ditto.buildDittoProtocolMsg(thingId[0], thingId[1], 'things', 'twin', 'commands', 'modify', '/features', headers, value);}", "outgoingScript": "function mapFromDittoProtocolMsg(namespace, id, group, channel, criterion, action, path, dittoHeaders, value, status, extra) {return null;}", "loadBytebufferJS": "false", "loadLongJS": "false" } } } } }

[Daedaluz]

could you provide mosquitto config?

[Gauravmeenu]

Yeah sure this is my mosquitto.conf file.

listener 1883 listener 9001 protocol websockets allow_anonymous true

[Daedaluz]

sorry, yea.. forgot to ask for mosquitto logs too.

[Gauravmeenu]

This is mosquitto logs.

1679896571: mosquitto version 2.0.15 starting 1679896571: Config loaded from /mosquitto/config/mosquitto.conf. 1679896571: Opening ipv4 listen socket on port 1883. 1679896571: Opening ipv6 listen socket on port 1883. 1679896571: Opening websockets listen socket on port 9001. 1679896571: mosquitto version 2.0.15 running

[Daedaluz]

ugh.. coffee hasn't kicked in. your config looks like it's configured to tcp://localhost:1883.

simply put; all containers has their own localhost (unless net is set to host). you need to have a testconnection config to reach the mosquitto container in some appropriate way.

localhost for where you have testconnection != localhost for mosquitto,

[Gauravmeenu]

Okay, so could you please help me to how I can my testconnection confiig to reach the mosquitto dokcer container .

[Daedaluz]

easiest would probably to just throw in the mosquitto service into the deployment file of ditto, then use tcp://mosquitto:1883 as url or similar.

[Gauravmeenu]

Sorry @Daedaluz I can't get you .

[gunjan-it-engg]

easiest would probably to just throw in the mosquitto service into the deployment file of ditto, then use tcp://mosquitto:1883 as url or similar.

Actually I also got the same issue , can you please elaborate more about this

[Gauravmeenu]

easiest would probably to just throw in the mosquitto service into the deployment file of ditto, then use tcp://mosquitto:1883 as url or similar.

I did this but it gives me a null response error.

[Daedaluz]

how does your docker-compose file look like now then?

[Gauravmeenu]

My docker-compose file looks like this:

version: '3'

services: mongodb: image: docker.io/mongo:4.4.18 mem_limit: 256m restart: always networks: default: aliases:

• mongodb command: mongod --storageEngine wiredTiger --noscripting user: mongodb ports:

  • 27017:27017 environment: TZ: Europe/Berlin

  policies: image: docker.io/eclipse/ditto-policies:${DITTO_VERSION:-latest} mem_limit: 512m restart: always networks: default: aliases:

• ditto-cluster environment:

  • TZ=Europe/Berlin
  • BIND_HOSTNAME=0.0.0.0

    Set additional configuration options here appending JAVA_TOOL_OPTIONS: -Dditto.policies...

  • JAVA_TOOL_OPTIONS=-XX:ActiveProcessorCount=2 -XX:+ExitOnOutOfMemoryError -XX:+UseContainerSupport -XX:+UseStringDeduplication -Xss512k -XX:MaxRAMPercentage=50 -XX:+UseG1GC -XX:MaxGCPauseMillis=150 -Dakka.coordinated-shutdown.exit-jvm=on -Dakka.cluster.shutdown-after-unsuccessful-join-seed-nodes=180s -Dakka.cluster.failure-detector.threshold=15.0 -Dakka.cluster.failure-detector.expected-response-after=10s -Dakka.cluster.failure-detector.acceptable-heartbeat-pause=20s -Dakka.cluster.downing-provider-class=
  • MONGO_DB_HOSTNAME=mongodb

    in order to write logs into a file you can enable this by setting the following env variable

    the log file(s) can be found in /var/log/ditto directory on the host machine

    - DITTO_LOGGING_FILE_APPENDER=true

    only needed if DITTO_LOGGING_FILE_APPENDER is set

    volumes:

    - ditto_log_files:/var/log/ditto

    healthcheck: test: curl --fail hostname:8558/alive || exit 1 interval: 30s timeout: 15s retries: 4 start_period: 120s

  things: image: docker.io/eclipse/ditto-things:${DITTO_VERSION:-latest} mem_limit: 512m restart: always networks: default: aliases:

• ditto-cluster depends_on:

  • policies environment:
  • TZ=Europe/Berlin
  • BIND_HOSTNAME=0.0.0.0

    Set additional configuration options here appending JAVA_TOOL_OPTIONS: -Dditto.things...

  • JAVA_TOOL_OPTIONS=-XX:ActiveProcessorCount=2 -XX:+ExitOnOutOfMemoryError -XX:+UseContainerSupport -XX:+UseStringDeduplication -Xss512k -XX:MaxRAMPercentage=50 -XX:+UseG1GC -XX:MaxGCPauseMillis=150 -Dakka.coordinated-shutdown.exit-jvm=on -Dakka.cluster.shutdown-after-unsuccessful-join-seed-nodes=180s -Dakka.cluster.failure-detector.threshold=15.0 -Dakka.cluster.failure-detector.expected-response-after=10s -Dakka.cluster.failure-detector.acceptable-heartbeat-pause=20s -Dakka.cluster.downing-provider-class=
  • MONGO_DB_HOSTNAME=mongodb

    in order to write logs into a file you can enable this by setting the following env variable

    the log file(s) can be found in /var/log/ditto directory on the host machine

    - DITTO_LOGGING_FILE_APPENDER=true

    only needed if DITTO_LOGGING_FILE_APPENDER is set

    volumes:

    - ditto_log_files:/var/log/ditto

    healthcheck: test: curl --fail hostname:8558/alive || exit 1 interval: 30s timeout: 15s retries: 4 start_period: 120s

  things-search: image: docker.io/eclipse/ditto-things-search:${DITTO_VERSION:-latest} mem_limit: 512m restart: always networks: default: aliases:

• ditto-cluster depends_on:

  • policies environment:
  • TZ=Europe/Berlin
  • BIND_HOSTNAME=0.0.0.0

    Set additional configuration options here appending JAVA_TOOL_OPTIONS: -Dditto.search...

  • JAVA_TOOL_OPTIONS=-XX:ActiveProcessorCount=2 -XX:+ExitOnOutOfMemoryError -XX:+UseContainerSupport -XX:+UseStringDeduplication -Xss512k -XX:MaxRAMPercentage=50 -XX:+UseG1GC -XX:MaxGCPauseMillis=150 -Dakka.coordinated-shutdown.exit-jvm=on -Dakka.cluster.shutdown-after-unsuccessful-join-seed-nodes=180s -Dakka.cluster.failure-detector.threshold=15.0 -Dakka.cluster.failure-detector.expected-response-after=10s -Dakka.cluster.failure-detector.acceptable-heartbeat-pause=20s -Dakka.cluster.downing-provider-class=
  • MONGO_DB_HOSTNAME=mongodb

    in order to write logs into a file you can enable this by setting the following env variable

    the log file(s) can be found in /var/log/ditto directory on the host machine

    - DITTO_LOGGING_FILE_APPENDER=true

    only needed if DITTO_LOGGING_FILE_APPENDER is set

    volumes:

    - ditto_log_files:/var/log/ditto

    healthcheck: test: curl --fail hostname:8558/alive || exit 1 interval: 30s timeout: 15s retries: 4 start_period: 120s

  connectivity: image: docker.io/eclipse/ditto-connectivity:${DITTO_VERSION:-latest} mem_limit: 768m restart: always networks: default: aliases:

• ditto-cluster depends_on:

  • policies environment:
  • TZ=Europe/Berlin
  • BIND_HOSTNAME=0.0.0.0

    if connections to rabbitmq broker are used, you might want to disable ExitOnOutOfMemoryError, because the amqp-client has a bug throwing OOM exceptions and causing a restart loop

    Set additional configuration options here appending JAVA_TOOL_OPTIONS: -Dditto.connectivity...

  • JAVA_TOOL_OPTIONS=-XX:ActiveProcessorCount=2 -XX:+ExitOnOutOfMemoryError -XX:+UseContainerSupport -XX:+UseStringDeduplication -Xss512k -XX:MaxRAMPercentage=50 -XX:+UseG1GC -XX:MaxGCPauseMillis=150 -Dakka.coordinated-shutdown.exit-jvm=on -Dakka.cluster.shutdown-after-unsuccessful-join-seed-nodes=180s -Dakka.cluster.failure-detector.threshold=15.0 -Dakka.cluster.failure-detector.expected-response-after=10s -Dakka.cluster.failure-detector.acceptable-heartbeat-pause=20s -Dakka.cluster.downing-provider-class=
  • MONGO_DB_HOSTNAME=mongodb

    in order to write logs into a file you can enable this by setting the following env variable

    the log file(s) can be found in /var/log/ditto directory on the host machine

    - DITTO_LOGGING_FILE_APPENDER=true

    only needed if DITTO_LOGGING_FILE_APPENDER is set

    volumes:

    - ditto_log_files:/var/log/ditto

    healthcheck: test: curl --fail hostname:8558/alive || exit 1 interval: 30s timeout: 15s retries: 4 start_period: 120s

  gateway: image: docker.io/eclipse/ditto-gateway:${DITTO_VERSION:-latest} mem_limit: 512m restart: always networks: default: aliases:

• ditto-cluster depends_on:

  • policies ports:
  • "8081:8080" environment:
  • TZ=Europe/Berlin
  • BIND_HOSTNAME=0.0.0.0
  • ENABLE_PRE_AUTHENTICATION=true

    Set additional configuration options here appending JAVA_TOOL_OPTIONS: -Dditto.gateway.authentication.devops.password=foobar -Dditto.gateway...

  • JAVA_TOOL_OPTIONS=-XX:ActiveProcessorCount=2 -XX:+ExitOnOutOfMemoryError -XX:+UseContainerSupport -XX:+UseStringDeduplication -Xss512k -XX:MaxRAMPercentage=50 -XX:+UseG1GC -XX:MaxGCPauseMillis=150 -Dakka.coordinated-shutdown.exit-jvm=on -Dakka.cluster.shutdown-after-unsuccessful-join-seed-nodes=180s -Dakka.cluster.failure-detector.threshold=15.0 -Dakka.cluster.failure-detector.expected-response-after=10s -Dakka.cluster.failure-detector.acceptable-heartbeat-pause=20s -Dakka.cluster.downing-provider-class=

    in order to write logs into a file you can enable this by setting the following env variable

    the log file(s) can be found in /var/log/ditto directory on the host machine

    - DITTO_LOGGING_FILE_APPENDER=true

    You may use the environment for setting the devops password

    - DEVOPS_PASSWORD=foobar

    only needed if DITTO_LOGGING_FILE_APPENDER is set

    volumes:

    - ditto_log_files:/var/log/ditto

    healthcheck: test: curl --fail hostname:8558/alive || exit 1 interval: 30s timeout: 15s retries: 4 start_period: 120s

  swagger-ui: image: docker.io/swaggerapi/swagger-ui:v4.14.3 mem_limit: 32m restart: always environment:

  • QUERY_CONFIG_ENABLED=true volumes:
    • ../../documentation/src/main/resources/openapi:/usr/share/nginx/html/openapi:ro
    • ../../documentation/src/main/resources/images:/usr/share/nginx/html/images:ro
    • ./swagger3-index.html:/usr/share/nginx/html/index.html:ro command: nginx -g 'daemon off;'

  nginx: image: docker.io/nginx:1.21-alpine mem_limit: 32m restart: always volumes:

  • ./nginx.conf:/etc/nginx/nginx.conf:ro
  • ./nginx.htpasswd:/etc/nginx/nginx.htpasswd:ro
  • ./nginx-cors.conf:/etc/nginx/nginx-cors.conf:ro
  • ./mime.types:/etc/nginx/mime.types:ro
  • ./index.html:/etc/nginx/html/index.html:ro
  • ../../documentation/src/main/resources/images:/etc/nginx/html/images:ro
  • ../../documentation/src/main/resources/wot:/etc/nginx/html/wot:ro
  • ../../ui:/etc/nginx/html/ui:ro ports:
    • "${DITTO_EXTERNAL_PORT:-8080}:80" depends_on:
    • gateway
    • swagger-ui

  mosquitto: image: eclipse-mosquitto:2.0 container_name: mosquitto restart: always volumes:

  • ./mosquitto.conf:/mosquitto/config/mosquitto.conf ports:
  • "1883:1883"
  • "9001:9001"

volumes: ditto_log_files: driver: local driver_opts: type: none device: /var/log/ditto o: bind,uid=1000,gid=1000

[Daedaluz]

what does "null" error response mean? does this mean it works? (like "no error")

or is the url supposed to be a websocket url, hence the websocket configuration?

[gunjan-it-engg]

no it's not work

[Gauravmeenu]

what does "null" error response mean? does this mean it works? (like "no error")

or is the url supposed to be a websocket url, hence the websocket configuration?

No it didn't work I got this kind of error

{ "type": "devops.responses:errorResponse", "status": 503, "serviceName": "connectivity", "instance": "1f46323d92f2", "payload": { "status": 503, "error": "ask.error", "message": "Ask timed out on [ActorSelection[Anchor(akka://ditto-cluster/), Path(/system/sharding/connection)]] after [10000 ms]. Message of type [org.eclipse.ditto.connectivity.model.signals.commands.modify.TestConnection]. A typical reason for AskTimeoutException is that the recipient actor didn't send a reply.", "description": "Please retry the performed action in order to improve resiliency." } }

[Daedaluz]

websocket url didn't work either?

[Gauravmeenu]

websocket url didn't work either?

I think connection is establish with tcp://mosquitto:1833, but I got error with mosquitto-client sub-pub

Unable to connect (Lookup error.).

[gunjan-it-engg]

is there any update regarding this @Daedaluz @Gauravmeenu @thjaeckle