search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.25k stars 21.41k forks source link

Sending sensor data through hardware gateway #36133

Closed xelossan closed 5 years ago

xelossan commented 5 years ago

Hi, What if I cannot add required properties to device's message? In my case, there's a single third party gateway device that collects telemetry from multiple sensors and sends it as a JSON array to configured endpoint. I cannot modify the message. Is there a way to support this case without adding another gateway/web app or something between the existing gateway device and Digital Twins?

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

Alberto-Vega commented 5 years ago

@xelossan Thanks for reaching out! So it looks like you can not add logic to the gateway to modify the message?

xelossan commented 5 years ago

No, the gateway has its own message format that cannot be modified without changing its firmware, which cannot be done without manufacturer's help.

The message looks like this: {"d0f01843e9cd":{"Time":"7","Type":"Care T","Rssi":"-74","Name":"","Batt":"2.88","Alarms":"4","Temperature":"25.58"},"d0f01843e9cc":{"Time":"0","Type":"Care T","Rssi":"-89","Name":"","Batt":"3","Alarms":"0","Temperature":"9.60"}} so actually it's not even a JSON array, but a dictionary, where the key is sensor's hardware ID.

I'm pretty sure that most of hardware gateways has some proprietary message formats that end users may not be able to modify, so the question is: is there a way (or any plans to add it in future) to somehow preprocess the telemetry message on Digital Twins service side before running matchers and UDFs, something like a special UDF that fires for every message and can modify it so it has a correct format and every required property.

KingdomOfEnds commented 5 years ago

Hi @xelossan - Thank you for your great question! First things first, you'll absolutely want to upgrade your firmware to the latest spec regardless. Many vendors supply important security updates along with new features.

To answer the question regarding format interoperability:

Presently, messages that are submitted to ADT must conform to the Message entity specified in the Azure IOT SDK. Specifically, Device Client Message which must be sent through the Device Client namespace. There are a variety of protocols supported (MQTT, HTTP) and JSON is one of the message formats you can use (providing quite a bit of flexibility). You can, for instance, specify the contents of a message using a Map and add filterable Properties (that are intercepted and can be used for routing).

However, even in those circumstances, you'd still want to serialize your message into the appropriate format. One way to accomplish that is to submit the Gateway's telemetry to an API where it can be serialized using custom code and then send that further downstream. In fact, you may find that useful even with an update to your firmware. On the other hand, it's probably easier to accomplish that using custom programming on your device itself.

PS - I will make sure to relay your observations surrounding format interoperability to the PM and Engineering teams. Thanks!

xelossan commented 5 years ago

Thanks for your answer @KingdomOfEnds, in that case using additional API between the gateway and ADT is the way to go for us.

I've got one more question - where can I post issues related to ADT but not to any specific docs page? We've encountered some strange behavior: we have 4 sensors added to the same Device in ADT graph, and one of them is working correctly, but values from the rest are ignored with a message like: Ignoring telemetry received from Device Id '3b8449a8-4f88-454f-82dd-aca586afcea2' due to no identifiable Sensor Id. Sensor Hardware ID: [d0f01843e9cc].

But the sensor with this ID is defined: { "pollRate": 0, "spaceId": "ab817e89-2e79-4077-a77e-01d8bc393ea5", "deviceId": "3b8449a8-4f88-454f-82dd-aca586afcea2", "id": "7d7f99af-93a3-4e1b-bc6b-94ffbe574624", "portTypeId": 8, "dataUnitTypeId": 38, "dataTypeId": 18, "dataSubtypeId": 5, "typeId": 9, "hardwareId": "d0f01843e9cc" }

Alberto-Vega commented 5 years ago

@xelossan For product feedback (feature requests or similar) you can find the link called "This Product" at the footer of each doc in Azure docs image Another good source for development questions is StackOverflow or MSDN also you could open a support ticket if you have access to a support plan?

Alberto-Vega commented 5 years ago

@KingdomOfEnds Thanks for the help! @xelossan We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.