Tutorial 0-provision-azure-cluster errs on Config.ProvisionCluster()

[MaxWilson]

Hi,

Newbie here. I hit an issue when trying the first tutorial (0-provision-azure-cluster.fsx). When I execute the line

let deployment = Config.ProvisionCluster()

I get a bunch of successes but also an error:

Binding session to 'D:\code\rtg\MBraceAzure\HandsOnTutorial.FSharp../packages/MBrace.Azure/tools\Newtonsoft.Json.dll'...

Hyak.Common.CloudException: uuid:f38b9357-360a-4df7-ad06-be239ab83090;id=54627132017-09-29T20:13:14ZCreating an ACS enabled Service Bus namespace failed for the subscription ef4cb8f038de4ee88fbbdef156954514. For additional details refer to https://go.microsoft.com/fwlink/?linkid=852159&amp;clcid=0x409. at MBrace.Runtime.Utils.Retry.aux@33-3.Invoke(FSharpChoice2 _arg2) in C:\Users\eirik\Development\mbrace\MBrace.Core\src\MBrace.Runtime\Utils\Retry.fs:line 36 at Microsoft.FSharp.Control.AsyncBuilderImpl.args@825-1.Invoke(a a) at MBrace.Core.Internals.AsyncExtensions.Async.RunSync[T](FSharpAsync1 workflow, FSharpOption`1 cancellationToken) in C:\Users\eirik\Development\mbrace\MBrace.Core\src\MBrace.Core\Utils\AsyncExtensions.fs:line 101 at <StartupCode$FSI_0012>.$FSI_0012.main@() in D:\code\rtg\MBraceAzure\HandsOnTutorial.FSharp\0-provision-azure-cluster.fsx:line 33 Stopped due to error

The referenced link (https://go.microsoft.com/fwlink/?linkid=852159&amp;clcid=0x409) says something about a recent change to Azure functionality.

ACS Namespaces you create before June 30, 2017 will not be affected and will be fully supported, however, attempts to create new ACS enabled Service Bus, Event Hubs, or Relay namespaces using PowerShell, ARM templates, and our APIs after June 30 will fail.

This failure blocks the rest of the tutorial, because calls to Config.GetCluster() now fail too.

[dsyme]

@MaxWilson Yes, unfortunately this means the tutorial is no longer valid, and there is no simple getting-started path for MBrace at the moment. We will need to adjust the docs until this is resolved

[MaxWilson]

Thanks. At least I don't have to bang my head on the wall now. :) My aim is familiarization and experimentation, so I guess I'll just browse the samples and the docs for now without running them, at least in Azure. Hopefully I can get the Thespian samples working in VS 2015...

[isaacabraham]

@MaxWilson In the short term, you can use the Thespian (local) cluster to get up and running. You could also give https://github.com/isaacabraham/mbrace-arm a try. @dsyme I'll endeavour to at least find out what's causing this over the coming days.

[laygr]

I ran into the same issue (I suppose many will do) and I was about to give up, but trying MBrace with Thespian sounds better.

[roguetrainer]

Same here! (Namely newbie seeing similar errors.) Glad it's not (only) my stupidity, & that I found this issue. All working fine with Thespian, so that's good.

I also tried manually creating from the Azure portal, three items:

1. Cloud Service (classic)
2. Storage Account
3. Service Bus,

taking note of the connection strings (for 2 & 3) & then submitting the commands here: http://mbrace.io/starterkit/HandsOnTutorial.FSharp/going-further/200-managing-azure-clusters.html namely:

let serviceBusConnection = " ... enter ServiceBusConnectionString here ..." 
let storageConnection = "... enter StorageConnectionString here ... "
let config = Configuration(storageConnection, serviceBusConnection) 
let cluster = AzureCluster.Connect(config, logger = ConsoleLogger(true), logLevel = LogLevel.Info)

Should that work at this point? Let me just get you the error log... wow, no errors this time. It worked! I have a cluster! :) But no workers. :( When I try to run 1-hello-world, I get

[2017-11-09 14:46:45] WARNING : Could not serialize data dependencies: MBrace.Azure.Management.PublishSettings pubSettings@, MBrace.Azure.Management.Subscription subscription@, MBrace.Azure.Management.SubscriptionManager manager@, MBrace.Azure.AzureCluster cluster@, MBrace.Azure.Management.PublishSettings pubSettings@, MBrace.Azure.Management.Subscription subscription@, MBrace.Azure.Management.SubscriptionManager manager@
[2017-11-09 14:46:45] **WARNING : No worker instances currently associated with cluster. Computation may never complete.**
[2017-11-09 14:46:45] INFO : Creating cloud process "xxxxxx"
[2017-11-09 14:46:46] INFO : Posted CloudProcess<string> 'xxxxxx'.

OK, probably best to look at the new ARM code that @isaacabraham is working on rather than work with the old code? Really looking forward to getting familiar with MBrace. This is a brilliant & powerful tool.

--

[roguetrainer]

In case this is useful to anyone else getting familiar with MBrace, I have been successful in creating four artifacts (visible on the Azure Portal) using the instructions in the README.md file, to execute commands in a PowerShell window

## Logs into Azure using your credentials - no need for pub setting 
Login-AzureRmAccount

## Create a "resource group" for the cluster, called MyCluster
New-AzureRmResourceGroup -Name MyCluster -Location "canadacentral"

## Create a 'small' MBrace cluster called MyCluster!
New-AzureRmResourceGroupDeployment -ResourceGroupName MyCluster -TemplateFile .\azuredeploy.json -TemplateParameterFile "small-cluster.json"

Bravo! (I see ProvisioningState : Succeeded.) The four resources created, which are visible from the Azure Portal, are App Service Plan, App Service, Storage Account and Service Bus.

However, I still don't have any workers? (On the basis of running cluster.ShowWorkers(), but perhaps that is a permissions issue?)

As above, I tried the F# commands here: 200-managing-azure-clusters.html namely:

let serviceBusConnection = " ... enter ServiceBusConnectionString here ..." 
let storageConnection = "... enter StorageConnectionString here ... "
let config = Configuration(storageConnection, serviceBusConnection) 
let cluster = AzureCluster.Connect(config, logger = ConsoleLogger(true), logLevel = LogLevel.Info)

which successfully gives me a cluster, but not a deployment? As before I get the warning: "WARNING : No worker instances currently associated with cluster. Computation may never complete."

At that point, should I be able to go straight to 1-hello-world.fsx & start to execute cloud workflows, such as

let quickText = 
    cloud { return "Hello world!" } 
    |> cluster.Run

or am I missing extra steps (to create workers)? Perhaps I should not expect any code from the HandsOnTutorial to work at this point during the transition to ARM? I hope these comments are useful for other people trying to do the same thing.

[isaacabraham]

Hi. No, that should be all you need to get things working for the ARM-based deployment. What version of VS are you running with?

Also - try getting worker stats from your cluster - what do you see?

The tutorials should work 100% with both ARM + App Service or the old Cloud Service approach. The only real different between them is that the ARM version hosts MBrace in the App Service (IIS) rather than Cloud Services (which are really just VMs with some add-ons). The code should work on both without an issue.

[o1lo01ol1o]

@roguetrainer I was able to provision and deploy a cluster by manually creating a ServiceBus resource and then passing connection strings for storage and the new bus:

let deployment = Deployment.Provision(pubSettingsFile, Region.East_US, 2, VMSize.A1, logger = ConsoleLogger(true), logLevel = LogLevel.Info, subscriptionId=subscription.Id, storageAccount=storageConnection, serviceBusAccount=serviceBusConnection)

There are some issues with VM sizes on for the smaller tiers and the provisioning takes a little bit of time before you'll get workers.