This repository contains complete and easy to use examples that demonstrate the power of Apache Storm on Microsoft Azure HDInsight.
The .Net examples are applicable to all HDInsight Windows Storm clusters.
The Java examples are applicable to :
For version 3.5+ Linux based HDInsight Storm clusters please look under HDI3.5 folder.
Official page - Apache Storm for Microsoft Azure HDInsight
Getting Started page - Getting Started with HDInsight Storm
Each of the examples folder contains a run.bat
that does the following:
prepare.ps1
)
build.ps1
)
execute.ps1
)
An existing run configuration is first detected to provide idem-potency in creation of Azure resources. This allows you to re-run the run.bat
in case of some unexpected failure.
If you just want to build the source code you can use the build.bat
in each example folder to build that example.
If you modified some source and wish to re-deploy the topologies, you can build using build.bat
and submit the topologies using execute.ps1
There are multiple end-to-end examples included in this repository that showcase different topologies that you can build and run against HDInsight Storm. HDInsight provides users the option to write applications in language of their choice and exposes Storm REST APIs as well as additional web interfaces for remote topology submission and management.
An internet of things example that demonstrates connected cars scenarios. The example showcases the reading of vehicle events from Event Hub, referencing Azure DocumentDB for vehicular data and then finally storing the computations in Azure Storage using the WASB driver on HDInsight clusters.
Read more about this example here: IoT example using Azure EventHubs, Storm, Azure DocumentDB, Azure Storage/WASB
A scalability benchmark that demonstrates throughput from EventHub/Kafka and stores the event counts in SQL Azure.
Read more about this example here: Event Count example using Azure EventHubs, Storm, SQL Azure
A real time ETL example with EventHubs, Storm and HBase that has two topologies.
Read more about this example here: Real Time ETL example using Azure EventHubs, Storm, HBase & SCP.Net
For writing Storm applications in C# using SCP.Net refer SCPNet-GettingStarted.md
Check out more SCP.Net Examples
This directory now contains many SCP.Net examples that show the APIs available in SCP.Net NuGet package. Follow the instructions in the above directory to deploy these examples to your HDInsight Storm cluster.
The HybridTopologyHostMode
in SCPNetExamples/HybridTopologyHostMode has examples on all the different types of hybrid modes.
Templates to connect to various Azure services: