page_type: sample languages:

• csharp
• powershell
• dockerfile products:
• azure-iot-hub
• azure-event-hubs
• azure-container-instances

• azure-container-registry name: "Azure IoT Device Telemetry Simulator" urlFragment: azure-iot-device-telemetry-simulator description: "The IoT Telemetry Simulator allows you to test Azure IoT Hub, Event Hub or Kafka ingestion at scale."

Azure IoT Device Telemetry Simulator

The IoT Telemetry Simulator allows you to test Azure IoT Hub, Event Hub or Kafka ingestion at scale. The implementation is communicating with Azure IoT Hub using multiplexed AMQP connections. An automation library allows you to run it as load test as part of a CI/CD pipeline.

A single AMQP connection can handle approximately 995 devices.

Quick Start

The quickest way to generate telemetry is using Docker with the following command:

docker run -it -e "IotHubConnectionString=HostName=your-iothub-name.azure-devices.net;SharedAccessKeyName=device;SharedAccessKey=your-iothub-key" mcr.microsoft.com/oss/azure-samples/azureiot-telemetrysimulator

The simulator expects the devices to already exist in Azure IoT Hub. If you need help creating simulation devices in an Azure IoT Hub use the included project IotSimulatorDeviceProvisioning or the Docker image:

docker run -it -e "IotHubConnectionString=HostName=your-iothub-name.azure-devices.net;SharedAccessKeyName=registryReadWrite;SharedAccessKey=your-iothub-key" -e DeviceCount=1000 mcr.microsoft.com/oss/azure-samples/azureiot-simulatordeviceprovisioning

Simulator input parameters

The amount of devices, their names and telemetry generated can be customized using parameter. The list below contains the supported configuration parameters:

Telemetry template

The simulator is able to create user customizable telemetry with dynamic variables (random, counter, time, unique identifier, value range).

To generate a custom piece of telemetry it is required to set the template and, optionally, variables.

The template defines how the telemetry looks like, having placeholders for variables. Variables are declared in the telemetry as $.VariableName. The optional header defines properties that will be transmitted as message properties.

Variables are declared defining how values in the template will be resolved.

Built-in variables

The following variables are provided out of the box:

Customizable variables

Customizable variables can be created with the following properties:

Example 1: Telemetry with temperature between 23 and 25 and a counter starting from 100

Template:

{ "deviceId": "$.DeviceId", "rand_int": $.Temp, "rand_double": $.DoubleValue, "Ticks": $.Ticks, "Counter": $.Counter, "time": "$.Time" }

Variables:

[{"name": "Temp", "random": true, "max": 25, "min": 23}, {"name":"Counter", "min":100, "max":102}, {"name": "DoubleValue", "randomDouble":true, "min":0.22, "max":1.25}]

Output:

{ "deviceId": "sim000001", "rand_int": 23, "rand_double": 0.207759137669466, "Ticks": 637097550115091350, "Counter": 100, "time": "2019-11-19T10:10:11.5091350Z" }
{ "deviceId": "sim000001", "rand_int": 23, "rand_double": 1.207232427664231, "Ticks": 637097550115952079, "Counter": 101, "time": "2019-11-19T10:10:11.5952079Z" }
{ "deviceId": "sim000001", "rand_int": 24, "rand_double": 0.992871827638167, "Ticks": 637097550116627320, "Counter": 102, "time": "2019-11-19T10:10:11.6627320Z" }
{ "deviceId": "sim000001", "rand_int": 24, "rand_double": 0.779288272162327, "Ticks": 637097550117027320, "Counter": 100, "time": "2019-11-19T10:10:11.7027320Z" }

Running with Docker:

docker run -it -e "IotHubConnectionString=HostName=your-iothub-name.azure-devices.net;SharedAccessKeyName=device;SharedAccessKey=your-iothub-key" -e Template="{ \"deviceId\": \"$.DeviceId\", \"rand_int\": $.Temp, \"rand_double\": $.DoubleValue, \"Ticks\": $.Ticks, \"Counter\": $.Counter, \"time\": \"$.Time\" }" -e Variables="[{name: \"Temp\", \"random\": true, \"max\": 25, \"min\": 23}, {\"name\":\"Counter\", \"min\":100, \"max\":102} ]" mcr.microsoft.com/oss/azure-samples/azureiot-telemetrysimulator

calling from PowerShell:

docker run -it -e "IotHubConnectionString=HostName=your-iothub-name.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=your-iothub-key" -e Template="{ \"""deviceId\""": \"""$.DeviceId\""", \"""rand_int\""": $.Temp, \"""rand_double\""": $.RandomDouble , \"""Ticks\""": $.Ticks, \"""Counter\""": $.Counter, \"""time\""": \"""$.Time\""", \"""engine\""": \"""$.Engine\""" }" -e Variables="[{name: \"""Temp\""", \"""random\""": true, \"""max\""": 25, \"""min\""": 23}, {\"""name\""":\"""Counter\""", \"""min\""":100, \"""max\""":102}, {name:\"""Engine\""", values: [\"""on\""", \"""off\"""]}]" -e DeviceCount=1 -e MessageCount=3 mcr.microsoft.com/oss/azure-samples/azureiot-telemetrysimulator

Example 2: Adding the engine status ("on" or "off") to the telemetry

Template:

{ "deviceId": "$.DeviceId", "rand_int": $.Temp, "Ticks": $.Ticks, "Counter": $.Counter, "time": "$.Time", "engine": "$.Engine" }

Variables:

[{"name": "Temp", "random": true, "max": 25, "min": 23}, {"name":"Counter", "min":100}, {"name": "Engine", "values": ["on", "off"]}]

Output:

{ "deviceId": "sim000001", "rand_int": 23, "Ticks": 637097644549666920, "Counter": 100, "time": "2019-11-19T12:47:34.9666920Z", "engine": "off" }
{ "deviceId": "sim000001", "rand_int": 24, "Ticks": 637097644550326096, "Counter": 101, "time": "2019-11-19T12:47:35.0326096Z", "engine": "on" }

Running with Docker:

docker run -it -e "IotHubConnectionString=HostName=your-iothub-name.azure-devices.net;SharedAccessKeyName=device;SharedAccessKey=your-iothub-key" -e Template="{ \"deviceId\": \"$.DeviceId\", \"rand_int\": $.Temp, \"Ticks\": $.Ticks, \"Counter\": $.Counter, \"time\": \"$.Time\", \"engine\": \"$.Engine\" }" -e Variables="[{name: \"Temp\", \"random\": true, \"max\": 25, \"min\": 23}, {\"name\":\"Counter\", \"min\":100}, {name:\"Engine\", values: [\"on\", \"off\"]}]" mcr.microsoft.com/oss/azure-samples/azureiot-telemetrysimulator

Example 3: Using a configuration file to customize simulation

docker run -it -e "IotHubConnectionString=HostName=your-iothub-name.azure-devices.net;SharedAccessKeyName=device;SharedAccessKey=your-iothub-key" -e "File=/config_files/test4-config-multiple-internals-per-device.json" -e DeviceCount=3 --mount type=bind,source=$pwd\test\IotTelemetrySimulator.Test\test_files,target=/config_files,readonly mcr.microsoft.com/oss/azure-samples/azureiot-telemetrysimulator

Where the file content is:

{
  "Variables": [
    {
      "name": "DeviceSequenceValue1",
      "sequence": true,
      "values": [ "$.Counter", "$.Counter", "$.Counter", "$.Counter", "$.Counter", "true", "false", "$.Counter" ]
    },
    {
      "name": "Device1Tags",
      "sequence": true,
      "values": [ "['ProducedPartCount']", "['ProducedPartCount']", "['ProducedPartCount']", "['ProducedPartCount']", "['ProducedPartCount']", "['Downtime']", "['Downtime']", "['ProducedPartCount']" ]
    },
    {
      "name": "Device1Downtime",
      "values": [ "true", "true", "true", "true", "false" ]
    },
    {
      "name": "Counter"
    }
  ],
  "Intervals": {
    "sim000001": 10000,
    "sim000002": [ 100 , 200 ]
  },
  "Payloads": [
    {
      "type": "template",
      "deviceId": "sim000001",
      "template": "{\"device\":\"$.DeviceId\",\"value\":\"$.DeviceSequenceValue1\",\"tags\": $.Device1Tags}"
    },
    {
      "type": "fix",
      "deviceId": "sim000002",
      "value": "{\"value\":\"myfixvalue\"}"
    },
    {
      "type": "template",
      "deviceId": "sim000003",
      "template": "{\"device\":\"$.DeviceId\",\"a\":\"b\",\"value\":\"$.DeviceSequenceValue1\"}"
    }
  ]
}

Generating high volume of telemetry

In order to generate a constant high volume of messages a single computer might not be enough. This section describes two way to run the simulator to ingest a high volume of messages

Azure Container Instance

Azure has container instances which allow the execution of containers with micro billing. This repository has a PowerShell script that creates azure container instances in your subscription. Requirements are having az cli installed.

To start the simulator in a single container instance:

.\SimulatorCloudRunner.ps1

You will be asked to enter the Azure IoT Hub Connection string. After that, a resource group and one or more container instances will be created.

The cloud runner can be customized with the following parameters (as -ParameterName ParameterValue):

Kubernetes

This repository also contains a helm chart to deploy the simulator to a Kubernetes cluster. An example release with helm for 5000 devices in 5 pods:

helm install sims iot-telemetry-simulator\. --namespace iotsimulator --set iotHubConnectionString="HostName=xxxx.azure-devices.net;SharedAccessKeyName=iothubowner;SharedAccessKey=xxxxxxx" --set replicaCount=5 --set deviceCount=5000

Automation

The IotTelemtrySimulator.Automation .NET Core 2.1 library allows you to run the IoT Telemetry Simulator as part of a pipeline or any other automation framework. See the automation guide.

Advanced options

• PartitionKey option for Event Hubs: When processing events downstream with a distributed engine, it is often more efficient to have related messages in the same partition. For example, if computing values over windows of events per device in Spark, having all the events for a given device in the same partition ensures they are received by a single compute node and ordered. On the other hand, the default round-robin behavior has advantages too, such as ensuring all partitions have equal load and that the system remains available even if a partition fails. Therefore, it is good to give the user control over the partition key.
• DuplicateEvery option to duplicate events: Upstream systems generating events usually offer at-least-once delivery guarantees. Given that there is no mechanism in IoT Hub / Event Hubs for distributed transactions, it is always possible for the client not to receive the response from the server after sending a message, or crashing before persisting the response, and therefore to retry sending an event. This results in message duplicates. When testing event processing solution, it's important to inject such behavior so as to test if/how the system reacts to it, e.g. by deduplication. The option DuplicateEvery allows randomly duplicating a fraction of messages, e.g. setting DuplicateEvery to 1000 will duplicate messages with a probability of 1/1000 for each, i.e. will duplicate on average every 1000th message. Note that messages may be duplicated more than once: a message will be duplicated N times with a probability of 1/(DuplicateEvery)^N.