Intermittent failure due to firewall rule not creating in time

[loopfish]

Hi,

I have been running your excellent task for some time now, and I do notice that occasionally, an instance of the task fails because the firewall rule has not finished creating, so the agent cannot reach the server. My scenario is that I have several instances of the task in one job, all run sequentially, not in parallel, and it only happens perhaps 1 in every 20 times. Is there a way to possibly customize the timeout of the firewall rule creation, to be able to increase it if necessary, or does the code already perform a watch on the rule creation, and if so, maybe that needs a bit of a delay?

Sam

[liprec]

Thanks for the feedback. I do recall that the official documentation states that it can take up to 10 minutes before the firewall rules are applied, but I have never seen that happening. And probably due to the changes Microsoft has made regarding COVID-19, it can take some time lately. I have seen this also with other APIs. Probably it is better to add a (small) configurable delay to the tasks.

[loopfish]

I've actually created a workaround - I set the start and end IP in your task to be 0.0.0.0, with 'delete after rule creation' ticked. I then have used parts of your task code to create a Powershell task before all your AAS tasks to add the IP of our custom agent to the firewall, followed by a 'Start-Sleep -s 30' pause to allow the rule to create:

#Connecting to Azure
Write-Output "Getting Azure account context"
$cred = New-Object System.Management.Automation.PSCredential ($env:ARM_CLIENT_ID, (ConvertTo-SecureString $(client_secret) -AsPlainText -Force))

Connect-AzAccount -ServicePrincipal -Credential $cred -TenantId $env:ARM_TENANT_ID -SubscriptionId $env:ARM_SUBSCRIPTION_ID

#Other sites to provide IPv4 public address with this type of request

<#
http://ipinfo.io/ip
http://ifconfig.me/ip
http://icanhazip.com
http://ident.me
http://smart-ip.net/myip
#>

$EnvironmentName = "xxxx" + $env:environment.ToLower() + "aas"
$ResourceGroup = "RG-" + $env:environment + "-BI"

#Setting additional parameters
$ExistingFirewallRuleName = 'DevOpsAgentIP'
$PubIPSource = 'ipinfo.io/ip'

$AServiceServer = Get-AzAnalysisServicesServer -Name $EnvironmentName -ResourceGroupName $ResourceGroup
$FirewallRules = ($AServiceServer).FirewallConfig.FirewallRules
$FirewallRuleNameList = $FirewallRules.FirewallRuleName
$powerBi = ($AServiceServer).FirewallConfig.EnablePowerBIService

#Getting previous IP from firewall rule, and new public IP
$PreviousRuleIndex = [Array]::IndexOf($FirewallRuleNameList, $ExistingFirewallRuleName)
$currentIP = (Invoke-WebRequest -uri $PubIPSource -UseBasicParsing).content.TrimEnd()
$previousIP = ($FirewallRules).RangeStart[$PreviousRuleIndex]

#Updating rules if request is coming from new IP address.
if (!($currentIP -eq $previousIP)) {
    Write-Output "Updating Analysis Service firewall config"
    $ruleNumberIndex = 1
    $Rules = @() -as [System.Collections.Generic.List[Microsoft.Azure.Commands.AnalysisServices.Models.PsAzureAnalysisServicesFirewallRule]]

    #Storing Analysis Service firewall rules
    $FirewallRules | ForEach-Object {
        $ruleNumberVar = "rule" + "$ruleNumberIndex"
        #Exception of storage of firewall rule is made for the rule to be updated
        if (!($_.FirewallRuleName -match "$ExistingFirewallRuleName")) {

            $start = $_.RangeStart
            $end = $_.RangeEnd
            $tempRule = New-AzAnalysisServicesFirewallRule `
                -FirewallRuleName $_.FirewallRuleName `
                -RangeStart $start `
                -RangeEnd $end

            Set-Variable -Name "$ruleNumberVar" -Value $tempRule
            $Rules.Add((Get-Variable $ruleNumberVar -ValueOnly))
            $ruleNumberIndex = $ruleNumberIndex + 1
        }
    }
    #Add rule for new IP
    $updatedRule = New-AzAnalysisServicesFirewallRule -FirewallRuleName "$ExistingFirewallRuleName" -RangeStart $currentIP -RangeEnd $currentIP

    $ruleNumberVar = "rule" + "$ruleNumberIndex"
    Set-Variable -Name "$ruleNumberVar" -Value $updatedRule
    $Rules.Add((Get-Variable $ruleNumberVar -ValueOnly))

    #Creating Firewall config object
    if ($powerBi) {
        $conf = New-AzAnalysisServicesFirewallConfig -FirewallRule $Rules -EnablePowerBiService
    }
    else {
        $conf = New-AzAnalysisServicesFirewallConfig -FirewallRule $Rules
    }

    #Setting firewall config
    if ([String]::IsNullOrEmpty($AServiceServer.BackupBlobContainerUri)) {
        $AServiceServer | Set-AzAnalysisServicesServer -FirewallConfig $conf -DisableBackup -Sku $AServiceServer.Sku.Name.TrimEnd()
    }
    else {
        $AServiceServer | Set-AzAnalysisServicesServer -FirewallConfig $conf -BackupBlobContainerUri $AServiceServer.BackupBlobContainerUri -Sku $AServiceServer.Sku.Name.TrimEnd()

    }
    Write-Output "Updated firewall rule to include current IP: $currentIP"
}
elseif ($currentIP -eq $previousIP) {
    Write-Output "Firewall rule for IP $currentIP already exists."
}

There is then a Powershell task, again based on your code, after all your tasks to remove that IP:

#Connecting to Azure
Write-Output "Getting Azure account context"
$cred = New-Object `
    System.Management.Automation.PSCredential ($env:ARM_CLIENT_ID, (ConvertTo-SecureString $(client_secret) -AsPlainText -Force))

Connect-AzAccount -ServicePrincipal -Credential $cred -TenantId $env:ARM_TENANT_ID -SubscriptionId $env:ARM_SUBSCRIPTION_ID

#Other sites to provide IPv4 public address with this type of request

<#
http://ipinfo.io/ip
http://ifconfig.me/ip
http://icanhazip.com
http://ident.me
http://smart-ip.net/myip
#>

$EnvironmentName = "xxx" + $env:environment.ToLower() + "aas"
$ResourceGroup = "RG-" + $env:environment + "-BI"

#Setting additional parameters
$ExistingFirewallRuleName = 'DevOpsAgentIP'
$PubIPSource = 'ipinfo.io/ip'

$AServiceServer = Get-AzAnalysisServicesServer -Name $EnvironmentName -ResourceGroupName $ResourceGroup
$FirewallRules = ($AServiceServer).FirewallConfig.FirewallRules
$FirewallRuleNameList = $FirewallRules.FirewallRuleName
$powerBi = ($AServiceServer).FirewallConfig.EnablePowerBIService

#Getting previous IP from firewall rule, and new public IP
$PreviousRuleIndex = [Array]::IndexOf($FirewallRuleNameList, $ExistingFirewallRuleName)
$currentIP = (Invoke-WebRequest -uri $PubIPSource -UseBasicParsing).content.TrimEnd()
$previousIP = ($FirewallRules).RangeStart[$PreviousRuleIndex]

#Updating rules if request is coming from existing IP address.
if  ($currentIP -eq $previousIP) {
    Write-Output "Updating Analysis Service firewall config"
    $ruleNumberIndex = 1
    $Rules = @() -as [System.Collections.Generic.List[Microsoft.Azure.Commands.AnalysisServices.Models.PsAzureAnalysisServicesFirewallRule]]

    #Storing Analysis Service firewall rules
    $FirewallRules | ForEach-Object {
        $ruleNumberVar = "rule" + "$ruleNumberIndex"
        #Exception of storage of firewall rule is made for the rule to be removed
        if (!($_.FirewallRuleName -match "$ExistingFirewallRuleName")) {

            $start = $_.RangeStart
            $end = $_.RangeEnd
            $tempRule = New-AzAnalysisServicesFirewallRule `
                -FirewallRuleName $_.FirewallRuleName `
                -RangeStart $start `
                -RangeEnd $end

            Set-Variable -Name "$ruleNumberVar" -Value $tempRule
            $Rules.Add((Get-Variable $ruleNumberVar -ValueOnly))
            $ruleNumberIndex = $ruleNumberIndex + 1
        }
    }

    #Creating Firewall config object
    if ($powerBi) {
        $conf = New-AzAnalysisServicesFirewallConfig -FirewallRule $Rules `
        -EnablePowerBiService
    }
    else {
        $conf = New-AzAnalysisServicesFirewallConfig -FirewallRule $Rules
    }

    #Setting firewall config
    if ([String]::IsNullOrEmpty($AServiceServer.BackupBlobContainerUri)) {
        $AServiceServer | Set-AzAnalysisServicesServer -FirewallConfig $conf -DisableBackup -Sku $AServiceServer.Sku.Name.TrimEnd()
    }
    else {
        $AServiceServer | Set-AzAnalysisServicesServer -FirewallConfig $conf -BackupBlobContainerUri $AServiceServer.BackupBlobContainerUri -Sku $AServiceServer.Sku.Name.TrimEnd()

    }
    Write-Output "Updated firewall to remove rule '$ExistingFirewallRuleName'."
}

I have not seen the failures since implementing this.

[liprec]

Probably I will add something like Start-Sleep -s 30 to the tasks :-)

[stale[bot]]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.