search

Azure / doAzureParallel

A R package that allows users to submit parallel workloads in Azure
MIT License
107 stars 51 forks source link

AzureRMR interface #345

Open hongooi73 opened 5 years ago

hongooi73 commented 5 years ago

@brnleehng Would you be interested in a PR to add an AzureRMR interface, now that it's on CRAN? This should simplify the process of setup, and also allow possible future integration with AzureStor for storage accounts.

brnleehng commented 5 years ago

Hi Hong,

Are you thinking of a PR for Batch management APIs to AzureRMR?

Thanks, Brian

hongooi73 commented 5 years ago

No, I mean adding an AzureRMR interface to doAzureParallel and/or rAzureBatch. AzureRMR provides the framework for interacting with Resource Manager, which can be extended and customised by other packages.

This means people could do something like

library(rAzureBatch)
az <- AzureRMR::get_azure_login()
res <- az$
    get_subscription("sub_id")$
    get_resource_group("resgrp")$
    create_batch("clusname", nodes=10, size="Standard_DS2_v2", ...etc...)

# get the batch cluster
clus <- res$get_cluster()
clus$start()
clus$stop()
# etc

The closest analogue might be the AzureContainers package, which has methods for creating and working with AKS clusters. See the code here for the methods to create the cluster. There is a vignette that describes how you would interact with the cluster (scroll down to the "deploy to a Kubernetes cluster" section).

hongooi73 commented 5 years ago

Here is a reference arch repo that demonstrates using AKS to provide a real-time scoring service in R.

https://github.com/Azure/RealtimeRDeployment