custom .cspkg deployment but can't give .cscfg -> can't customize Network Configuration

[smoothdeveloper]

I'm trying to setup my mbrace cluster in a specific virtual network.

I've found in Azure documentation that this is defined in with <NetworkConfiguration/> element.

<NetworkConfiguration>
    <VirtualNetworkSite name="MyVirtualNetworkName"/>
        <AddressAssignments>
            <InstanceAddress roleName="MBrace.Azure.WorkerRole">
                <Subnets>
                    <Subnet name="MySubnetName"/>
                </Subnets>
            </InstanceAddress>
        </AddressAssignments>
</NetworkConfiguration>

Doing so and packaging my own .cspkg, I was still facing issues with machines not being deployed in same virtual network, and even machines within same cloud service won't be able to open TCP connections between each-others (despite the ports I've defined in ServiceDefinition.csdef are correctly visible in portal).

I've looked at https://github.com/mbraceproject/MBrace.Azure/blob/be89d86a39627518bae73d4766633563dd1d1115/src/MBrace.Azure.Management/Compute.fs#L202 and found that this is actual configuration being passed while deploying mbrace cluster, and that a way to override this was maybe the solution to my problem.

I've made a fork with rough extension point to try out: https://github.com/smoothdeveloper/MBrace.Azure/commit/084e5dbcd4735effbf52310944243b5bd0bf8dca which I used and am confirming this is the solution to be able to specify arbitrary network configuration on deployment, as well as whatever else can be defined in ServiceConfiguration.Cloud.cscfg files.

Do you see any other way to be able to pass my own config? Should I make a PR from my custom branch and if so what do you want me to modify?

Thanks!

[isaacabraham]

Nice, that should work. My only concern is that having an output as a string is a bit "black magic". Can we change this somehow to make it a bit more "self documenting" (without resorting to making an entire object model for the XML schema!)?

[isaacabraham]

I do wonder though if it's worth creating an IAAS version of MBrace which is just a raw VM as an ARM template using DSC to download and install the MBrace Azure exe somewhere (somehow).

[smoothdeveloper]

@isaacabraham I'm proposing, unless we have a clear idea of alternative, we make a very rough extension point for now; maybe I should define a DU with those two cases for now:

type CloudServiceConfigurationMaker = 
    | Default // this will be default value for the argument, and rely on existing internal function
    | MakeCloudServiceConfigurationXmlString of (the whole signature I was giving)

This hopefully gives me place to put detailed documentation.

Returning the string is what is used in underlying implementation and that configuration file is also a 1-1 to what is in MBrace.StarterKit/azure so I guess it is necessary to have at least this one way to do it (even if we can add other DU cases for other use cases).

Please let me know what you think of this and I'll adjust my branch / submit a PR.

I do wonder though if it's worth creating an IAAS version of MBrace which is just a raw VM as an ARM template using DSC to download and install the MBrace Azure exe somewhere (somehow).

My main concern so far is that provisioning the machines is very slow, I'm integrating mbrace in my application and user has to wait for cluster to be provisioned etc.

I haven't compared in detail how it goes if I have already provisioned VM that are stopped but that was also slow.

Do you see anyway this could improve with what you are saying?

[smoothdeveloper]

Updating my code and made PR #156, please let me know if this sounds like something we can integrate?

[isaacabraham]

We should have a chat about provisioning speed offline (or create a separate issue on this) - it goes to more than MBrace but more about the quickest ways of spinning up a compute engine of some sort in Azure.

Thanks for the PR as well - I'll review it when I get a chance (unless you want to have a look @eiriktsarpalis)