Connection between Hono and Ditto - New tenant - Failed

[paulocabrita-ionos]

Hello everyone.

I'm following the tour: https://www.eclipse.org/packages/packages/cloud2edge/tour/ and my aim is to create a new tenant, create some devices, send data to Hono and manipulate Ditto. Everything works fine except that Ditto does not receive any data.

I also noticed that the site has instructions to create a new connection between Hono and Ditto using AMQP but... the example that is also deployed uses Kafka. The example uses the domain "org.eclipse.packages.c2e".

If I had devices to the "org.eclipse.packages.c2e" domain everything works fine.

My curls are the following:

• Tenant curl -i -X POST http://${REGISTRY_IP}:${REGISTRY_PORT_HTTP}/v1/tenants/skynet
• New device curl -i -X POST http://${REGISTRY_IP}:${REGISTRY_PORT_HTTP}/v1/devices/skynet/skynet:my-device-1
• Device password curl -i -X PUT -H "Content-Type: application/json" --data '[ { "type": "hashed-password", "auth-id": "userRobot", "secrets": [{ "pwd-plain": "topSecret" }] }]' http://${REGISTRY_IP}:${REGISTRY_PORT_HTTP}/v1/credentials/skynet/skynet:my-device-1
• AMQP Connection (following the "tour"). HONO_TENANT and DITTO_DEVOPS_PWD are correct. curl -i -X POST -u devops:${DITTO_DEVOPS_PWD} -H 'Content-Type: application/json' --data '{ "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:createConnection", "connection": { "id": "hono-connection-for-'"${HONO_TENANT}"'", "connectionType": "amqp-10", "connectionStatus": "open", "uri": "amqp://consumer%40HONO:verysecret@'"${RELEASE}"'-dispatch-router-ext:15672", "failoverEnabled": true, "sources": [ { "addresses": [ "telemetry/'"${HONO_TENANT}"'", "event/'"${HONO_TENANT}"'" ], "authorizationContext": [ "pre-authenticated:hono-connection" ], "enforcement": { "input": "{{ header:device_id }}", "filters": [ "{{ entity:id }}" ] }, "headerMapping": { "hono-device-id": "{{ header:device_id }}", "content-type": "{{ header:content-type }}" }, "replyTarget": { "enabled": true, "address": "{{ header:reply-to }}", "headerMapping": { "to": "command/'"${HONO_TENANT}"'/{{ header:hono-device-id }}", "subject": "{{ header:subject | fn:default(topic:action-subject) | fn:default(topic:criterion) }}-response", "correlation-id": "{{ header:correlation-id }}", "content-type": "{{ header:content-type | fn:default('"'"'application/vnd.eclipse.ditto+json'"'"') }}" }, "expectedResponseTypes": [ "response", "error" ] }, "acknowledgementRequests": { "includes": [], "filter": "fn:filter(header:qos,'"'"'ne'"'"','"'"'0'"'"')" } }, { "addresses": [ "command_response/'"${HONO_TENANT}"'/replies" ], "authorizationContext": [ "pre-authenticated:hono-connection" ], "headerMapping": { "content-type": "{{ header:content-type }}", "correlation-id": "{{ header:correlation-id }}", "status": "{{ header:status }}" }, "replyTarget": { "enabled": false, "expectedResponseTypes": [ "response", "error" ] } } ], "targets": [ { "address": "command/'"${HONO_TENANT}"'", "authorizationContext": [ "pre-authenticated:hono-connection" ], "topics": [ "_/_/things/live/commands", "_/_/things/live/messages" ], "headerMapping": { "to": "command/'"${HONO_TENANT}"'/{{ thing:id }}", "subject": "{{ header:subject | fn:default(topic:action-subject) }}", "content-type": "{{ header:content-type | fn:default('"'"'application/vnd.eclipse.ditto+json'"'"') }}", "correlation-id": "{{ header:correlation-id }}", "reply-to": "{{ fn:default('"'"'command_response/'"${HONO_TENANT}"'/replies'"'"') | fn:filter(header:response-required,'"'"'ne'"'"','"'"'false'"'"') }}" } }, { "address": "command/'"${HONO_TENANT}"'", "authorizationContext": [ "pre-authenticated:hono-connection" ], "topics": [ "_/_/things/twin/events", "_/_/things/live/events" ], "headerMapping": { "to": "command/'"${HONO_TENANT}"'/{{ thing:id }}", "subject": "{{ header:subject | fn:default(topic:action-subject) }}", "content-type": "{{ header:content-type | fn:default('"'"'application/vnd.eclipse.ditto+json'"'"') }}", "correlation-id": "{{ header:correlation-id }}" } } ] } } }' http://${DITTO_API_IP}:${DITTO_API_PORT_HTTP}/devops/piggyback/connectivity
• Creation of the Digital Twin. I used a json file to do it (I created before the policy) curl -u ditto:ditto -X PUT -H 'Content-Type: application/json' -d @../robotArm_v1.json http://${DITTO_API_IP}:${DITTO_API_PORT_HTTP}/api/2/things/skynet:my-device-1
• I send data but nothing gets to the Digital Twin;
• I repeated the same process but this time I created a Kafka connection like this (I'm not sure if it's correct) curl -i -X POST -u devops:${DITTO_DEVOPS_PWD} -H 'Content-Type: application/json' --data '{ "targetActorSelection": "/system/sharding/connection", "headers": { "aggregate": false }, "piggybackCommand": { "type": "connectivity.commands:createConnection", "connection": { "id": "hono-skynet", "name": "skynet", "connectionType": "kafka", "connectionStatus": "open", "uri": "ssl://ditto-c2e:verysecret@c2e-kafka:9092", "sources": [ { "addresses": [ "hono.telemetry.skynet" ], "consumerCount": 1, "qos": 0, "authorizationContext": [ "pre-authenticated:hono-connection" ], "enforcement": { "input": "{{ header:device_id }}", "filters": [ "{{ entity:id }}" ] }, "acknowledgementRequests": { "includes": [], "filter": "fn:delete()" }, "headerMapping": {}, "replyTarget": { "address": "hono.command.skynet/{{ thing:id }}", "headerMapping": { "device_id": "{{ thing:id }}", "subject": "{{ header:subject | fn:default(topic:action-subject) | fn:default(topic:criterion) }}-response", "correlation-id": "{{ header:correlation-id }}" }, "expectedResponseTypes": [ "response", "error" ], "enabled": true } }, { "addresses": [ "hono.event.skynet" ], "consumerCount": 1, "qos": 1, "authorizationContext": [ "pre-authenticated:hono-connection" ], "enforcement": { "input": "{{ header:device_id }}", "filters": [ "{{ entity:id }}" ] }, "acknowledgementRequests": { "includes": [] }, "headerMapping": {}, "replyTarget": { "address": "hono.command.skynet/{{ thing:id }}", "headerMapping": { "device_id": "{{ thing:id }}", "subject": "{{ header:subject | fn:default(topic:action-subject) | fn:default(topic:criterion) }}-response", "correlation-id": "{{ header:correlation-id }}" }, "expectedResponseTypes": [ "response", "error" ], "enabled": true } }, { "addresses": [ "hono.command_response.skynet" ], "consumerCount": 1, "qos": 0, "authorizationContext": [ "pre-authenticated:hono-connection" ], "enforcement": { "input": "{{ header:device_id }}", "filters": [ "{{ entity:id }}" ] }, "acknowledgementRequests": { "includes": [], "filter": "fn:delete()" }, "headerMapping": { "correlation-id": "{{ header:correlation-id }}", "status": "{{ header:status }}" }, "replyTarget": { "enabled": false } } ], "targets": [ { "address": "hono.command.skynet/{{ thing:id }}", "topics": [ "_/_/things/live/commands", "_/_/things/live/messages" ], "authorizationContext": [ "pre-authenticated:hono-connection" ], "headerMapping": { "device_id": "{{ thing:id }}", "subject": "{{ header:subject | fn:default(topic:action-subject) }}", "response-required": "{{ header:response-required }}", "correlation-id": "{{ header:correlation-id }}" } }, { "address": "hono.command.skynet/{{thing:id}}", "topics": [ "_/_/things/twin/events", "_/_/things/live/events" ], "authorizationContext": [ "pre-authenticated:hono-connection" ], "headerMapping": { "device_id": "{{ thing:id }}", "subject": "{{ header:subject | fn:default(topic:action-subject) }}", "correlation-id": "{{ header:correlation-id }}" } } ], "clientCount": 1, "failoverEnabled": true, "validateCertificates": true, "processorPoolSize": 1, "specificConfig": { "saslMechanism": "plain", "bootstrapServers": "c2e-kafka:9092", "groupId": "skynet_{{ connection:id }}" } } } }' http://${DITTO_API_IP}:${DITTO_API_PORT_HTTP}/devops/piggyback/connectivity
• When I check the connection, it seems that everything is ok curl -X POST -i -u devops:${DITTO_DEVOPS_PWD} -H 'Content-Type: application/json' -d '{ "targetActorSelection": "/system/sharding/connection", "headers": {}, "piggybackCommand": { "type": "connectivity.commands:retrieveConnection", "connectionId": "hono-skynet" } }' http://${DITTO_API_IP}:${DITTO_API_PORT_HTTP}/devops/piggyback/connectivity?timeout=8s

Like I said, if I use the tenant from the example "org.eclipse.packages.c2e", it works fine but now when I create a new tenant.

I think this a related issue:

• https://github.com/eclipse-ditto/ditto/issues/1574

Also, I think it should be great to update the "tour" site in order to have instructions to connect Hono and Ditto using Kafka.

Does anyone have a working new kafka connection running with a new tenant?

Please advise.

Thanks PJC

[thjaeckle]

Please move this ticket to the IoT Packages project, as this is scope of the packages project to provide this combination of Hono/Ditto: https://github.com/eclipse/packages

There is already a known issue that the "tour" should be updated to using Kafka, however noone contributed on that so far: https://github.com/eclipse/packages/issues/432

[paulocabrita-ionos]

Thanks.

I will create the new issue in the "packages".