Automate task to delete and recreate HD Insight clusters

[IronmanM]

Hello Team,

Can we create the Script for Automate task to delete/Stop and recreate/start for HD Insight clusters / or Head node, please share documents/ Microsoft article

Thank you.

[CHEEKATLAPRADEEP-MSFT-zz]

@IronmanM You can create and delete Azure HDInsight clusters using Azure Automation.

HDInsight can be managed in Azure Automation by using the Azure HDInsight cmdlets that are available in the Azure PowerShell tools. Azure Automation has these cmdlets available out of the box, so that you can perform your HDInsight management tasks within the service. You can also pair these cmdlets in Azure Automation with the cmdlets for other Azure services, to automate complex tasks across Azure services and 3rd party systems.

You may refer this article which explains how to create and delete HDInsight clusters using Azure Automation.

Hope this helps.

[CHEEKATLAPRADEEP-MSFT-zz]

@IronmanM We will now proceed to close this thread. If there are further questions regarding this matter, please comment and we will gladly continue the discussion.

[IronmanM]

Hello @CHEEKATLAPRADEEP-MSFT ,

Thanks for your information,

I have tried the below script to create the Cluster with automation. $Conn = Get-AutomationConnection -Name AzureRunAsConnection Add-AzureRMAccount -ServicePrincipal -Tenant $Conn.TenantID -ApplicationId $Conn.ApplicationID -CertificateThumbprint $Conn.CertificateThumbprint $subscriptionID = Get-AutomationVariable –Name ‘0b3079f7-d19c-400e-8793-7b678538fc37’ Select-AzureRmSubscription –SubscriptionId $subscriptionID $resourceGroup = "marvel" $storageAccount = "ironman" $containerName = "spiderman" $storageAccountKey = (Get-AzureRmStorageAccountKey –Name $storageAccount –ResourceGroupName $resourceGroup)[0].value $clusterName = "marvelcluster1" $clusterCreds = Get-AutomationPSCredential –Name 'cluster-admin' $sshCreds = Get-AutomationPSCredential –Name 'ssh-user' $clusterType = "Hadoop" $clusterOS = "Linux" $clusterWorkerNodes = 2 $clusterNodeSize = "Standard_D1v2" $location = Get-AzureRmStorageAccount –StorageAccountName $storageAccount –ResourceGroupName $resourceGroup | %{$.Location} New-AzureRmHDInsightCluster –ClusterName $clusterName –ResourceGroupName $resourceGroup –Location $location –DefaultStorageAccountName "$storageAccount.blob.core.windows.net" –DefaultStorageAccountKey $storageAccountKey -DefaultStorageContainer $containerName –ClusterType $clusterType –OSType $clusterOS –Version "3.5" –HttpCredential $clusterCreds –SshCredential $sshCreds –ClusterSizeInNodes $clusterWorkerNodes –HeadNodeSize $clusterNodeSize –WorkerNodeSize $clusterNodeSize

i'm getting the error please look into this. Error :

Environments

---

{[AzureChinaCloud, AzureChinaCloud], [AzureCloud, AzureCloud], [AzureGermanCloud, AzureGermanCloud], [AzureUSGovernme... Get-AutomationVariable : A positional parameter cannot be found that accepts argument '�0b3079f7-d19c-400e-8793-7b678538fc37�'. At line:3 char:19

• ... riptionID = Get-AutomationVariable �Name �0b3079f7-d19c-400e-8793-7b6 ...

• 
  + CategoryInfo          : InvalidArgument: (:) [Get-AutomationVariable], ParameterBindingException
  + FullyQualifiedErrorId : 
  PositionalParameterNotFound,Orchestrator.AssetManagement.Cmdlets.GetAutomationVariableCmdlet

Set-AzureRmContext : A positional parameter cannot be found that accepts argument '$null'. At line:4 char:1

• Select-AzureRmSubscription �SubscriptionId $subscriptionID

• 
  + CategoryInfo          : InvalidArgument: (:) [Set-AzureRmContext], ParameterBindingException
  + FullyQualifiedErrorId : PositionalParameterNotFound,Microsoft.Azure.Commands.Profile.SetAzureRMContextCommand

Get-AzureRmStorageAccountKey : A positional parameter cannot be found that accepts argument '�ResourceGroupName'. At line:8 char:23

• ... countKey = (Get-AzureRmStorageAccountKey �Name $storageAccount �Resou ...

• 
  + CategoryInfo          : InvalidArgument: (:) [Get-AzureRmStorageAccountKey], ParameterBindingException
  + FullyQualifiedErrorId : 
  PositionalParameterNotFound,Microsoft.Azure.Commands.Management.Storage.GetAzureStorageAccountKeyCommand

Get-AutomationPSCredential : A positional parameter cannot be found that accepts argument 'cluster-admin'. At line:10 char:17

• $clusterCreds = Get-AutomationPSCredential �Name 'cluster-admin'

• 
  + CategoryInfo          : InvalidArgument: (:) [Get-AutomationPSCredential], ParameterBindingException
  + FullyQualifiedErrorId : 
  PositionalParameterNotFound,Orchestrator.AssetManagement.Cmdlets.GetAutomationPSCredentialCmdlet

Get-AutomationPSCredential : A positional parameter cannot be found that accepts argument 'ssh-user'. At line:11 char:13

• $sshCreds = Get-AutomationPSCredential �Name 'ssh-user'

• 
  + CategoryInfo          : InvalidArgument: (:) [Get-AutomationPSCredential], ParameterBindingException
  + FullyQualifiedErrorId : 
  PositionalParameterNotFound,Orchestrator.AssetManagement.Cmdlets.GetAutomationPSCredentialCmdlet

Get-AzureRmStorageAccount : A positional parameter cannot be found that accepts argument '�ResourceGroupName'. At line:16 char:13

• $location = Get-AzureRmStorageAccount �StorageAccountName $storageAcc ...

• 
  + CategoryInfo          : InvalidArgument: (:) [Get-AzureRmStorageAccount], ParameterBindingException
  + FullyQualifiedErrorId : 
  PositionalParameterNotFound,Microsoft.Azure.Commands.Management.Storage.GetAzureStorageAccountCommand

New-AzureRmHDInsightCluster : A positional parameter cannot be found that accepts argument 'marvel'. At line:17 char:1

• New-AzureRmHDInsightCluster �ClusterName $clusterName �ResourceGroupN ...

• 
  + CategoryInfo          : InvalidArgument: (:) [New-AzureRmHDInsightCluster], ParameterBindingException
  + FullyQualifiedErrorId : 
  PositionalParameterNotFound,Microsoft.Azure.Commands.HDInsight.NewAzureHDInsightClusterCommand

Thank you.

[IronmanM]

For below Command : $storageAccountKey = (Get-AzureRmStorageAccountKey –Name $storageAccount –ResourceGroupName $resourceGroup)[0].value

Getting this Error: Get-AzureRmStorageAccountKey : Cannot validate argument on parameter 'Name'. The argument is null or empty. Provide an argument that is not null or empty, and then try the command again. At line:1 char:58

• ... ountKey = (Get-AzureRmStorageAccountKey –Name $storageAccount –Resour ...

• 
  + CategoryInfo          : InvalidData: (:) [Get-AzureRmStorageAccountKey], ParameterBindingValidationException
  + FullyQualifiedErrorId : 
  ParameterArgumentValidationError,Microsoft.Azure.Commands.Management.Storage.GetAzureStorageAccountKeyCommand

[IronmanM]

Image link : https://imgur.com/6FeYcsW

[CHEEKATLAPRADEEP-MSFT-zz]

@IronmanM Since this issue is not related directly to the doc, I would recommend you to create a thread on the forums - MSDN. Once you post your issue on forums, it will have visibility across the community which is a better suited audience for such types of issues. Please create a Forum thread and post it's link here and tag me. I will now proceed to close this issue. Feel free to comment if you have any other queries.