search

MicrosoftDocs / azure-docs

Open source documentation of Microsoft Azure
https://docs.microsoft.com/azure
Creative Commons Attribution 4.0 International
10.27k stars 21.45k forks source link

Filtering telemetry messages #34748

Closed obrookstein closed 5 years ago

obrookstein commented 5 years ago

Hey, IoT hub supports filtering income telemetry messages and routing them to different endpoints based on their header and body.

After reading the DT documentation back and forth, and trying different things, it seems that there’s no way of doing it in DT. One can only filter on the event type (device message/ custom UDF etc.)

In my scenario, I would like to filter different messages based on the sensor Id, to different event hubs. Is there a way of doing it? Am I missing something?

Thanks,

Document Details

⚠ Do not edit this section. It is required for docs.microsoft.com ➟ GitHub issue linking.

asergaz commented 5 years ago

Hello @obrookstein thank you for your feedback. You are correct, if you need to filter different messages based on the sensor Id, you will filter your messages using User Defined Functions (as described here) .

In order to route UDF events to Event Hub you will configure it as described here.

For eg:

{
  "type": "EventHub",
  "eventTypes": [
    "UdfCustom"
  ],
  "connectionString": "Endpoint=sb://YOUR_NAMESPACE.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=YOUR_PRIMARY_KEY;EntityPath=YOUR_EVENT_HUB_NAME",
  "secondaryConnectionString": "Endpoint=sb://YOUR_NAMESPACE.servicebus.windows.net/;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=YOUR_SECONDARY_KEY;EntityPath=YOUR_EVENT_HUB_NAME",
  "path": "YOUR_EVENT_HUB_NAME"
}

See also the related threads on MSDN:

Thank you!

obrookstein commented 5 years ago

Thanks @sergaz-msft,

I understand that I can setup different matchers, to route telemetry messages to different UDF. However, eventually, after the UDF has processed the data, all results will be sent to the same endpoint as I configured for UDFCustom - correct?

I mean that there's no way, to route/sort the telemetry, based on the sensor id / type for instance to different endpoint. For example - occupancy sensors' events goes to EventHub # 1 and Temp sensors' events goes to event hub # 2 - correct?

Thanks,

asergaz commented 5 years ago

I didn't test this scenario you are describing though I would say that you can achieve it by creating two different UDF functions?

obrookstein commented 5 years ago

@sergaz-msft - that was my understanding too, but after reading this article (https://docs.microsoft.com/en-us/azure/digital-twins/how-to-egress-endpoints#configure-endpoints), it seems that the routing is done by the event type, and not the specific UDF. this means, that you can ALL event of type udfCustom to the same endpoint, and cannot route different udf results to different end points.

alinamstanciu commented 5 years ago

Hi @obrookstein, All events of certain type (DeviceMessages, TopologyOperation, SpaceChange, SensorChange, UdfCustom) they could be configured to go to one endpoint or more. But you cannot filter by sensor Id or any twin information. Routing is based on event type currently. In the future we are looking to higher fidelity.

obrookstein commented 5 years ago

Thanks @alinamstanciu - exactly the answer I was looking for!

AshokPeddakotla-MSFT commented 5 years ago

@obrookstein We will now proceed to close this thread. If there are further questions regarding this matter, please tag me in your reply. We will gladly continue the discussion and we will reopen the issue.