Closed dvmorris closed 9 years ago
Hi Dave,
Yes, the IotSample shows how to write to Azure Blob Storage using the storm-hdfs project.
Thanks, Shanyu
From: Dave Morris [mailto:notifications@github.com] Sent: Monday, February 16, 2015 6:29 AM To: hdinsight/hdinsight-storm-examples Subject: [hdinsight-storm-examples] Sample code that writes to an Azure Blob (#1)
Do you have any sample code where Storm writes out to an Azure blob, rather than HBase? We are having trouble getting any examples to work that do not write to HBase.
— Reply to this email directly or view it on GitHubhttps://github.com/hdinsight/hdinsight-storm-examples/issues/1.
Could you send me a URL to that sample code? I cannot find it on GitHub.
The code will be pushed out in a day or so. I appreciate your patience.
Thanks, Shanyu
From: Dave Morris [mailto:notifications@github.com] Sent: Monday, February 16, 2015 2:32 PM To: hdinsight/hdinsight-storm-examples Cc: Shanyu Zhao Subject: Re: [hdinsight-storm-examples] Sample code that writes to an Azure Blob (#1)
Could you send me a URL to that sample code? I cannot find it on GitHub.
— Reply to this email directly or view it on GitHubhttps://github.com/hdinsight/hdinsight-storm-examples/issues/1#issuecomment-74582654.
Sounds good. Looking forward to seeing the code!
On Mon, Feb 16, 2015 at 4:54 PM, Shanyu Zhao notifications@github.com wrote:
The code will be pushed out in a day or so. I appreciate your patience.
Thanks, Shanyu
From: Dave Morris [mailto:notifications@github.com] Sent: Monday, February 16, 2015 2:32 PM To: hdinsight/hdinsight-storm-examples Cc: Shanyu Zhao Subject: Re: [hdinsight-storm-examples] Sample code that writes to an Azure Blob (#1)
Could you send me a URL to that sample code? I cannot find it on GitHub.
— Reply to this email directly or view it on GitHub< https://github.com/hdinsight/hdinsight-storm-examples/issues/1#issuecomment-74582654>.
— Reply to this email directly or view it on GitHub https://github.com/hdinsight/hdinsight-storm-examples/issues/1#issuecomment-74585189 .
Please take a look at: IotTopology.java in IotExample
//Create HdfsBolt to store data to Windows Azure Blob Storage
SyncPolicy syncPolicy = new CountSyncPolicy(1000);
FileRotationPolicy rotationPolicy = new FileSizeRotationPolicy(1.0f, Units.MB);
RecordFormat recordFormat = new DelimitedRecordFormat().withFieldDelimiter(",");
FileNameFormat fileNameFormat = new DefaultFileNameFormat().withPath("/iotexampledata/");
HdfsBolt wasbBolt = new HdfsBolt()
.withFsUrl("wasb:///")
.withRecordFormat(recordFormat)
.withFileNameFormat(fileNameFormat)
.withRotationPolicy(rotationPolicy)
.withSyncPolicy(syncPolicy);
topologyBuilder.setBolt("WasbStoreBolt", wasbBolt, 10)
.fieldsGrouping("DataReferenceBolt", new Fields("model")).setNumTasks(10);
Do you have any sample code where Storm writes out to an Azure blob, rather than HBase? We are having trouble getting any examples to work that do not write to HBase.