Azure / azure-functions-host

The host/runtime that powers Azure Functions
https://functions.azure.com
MIT License
1.92k stars 441 forks source link

ARM Template - Automate Azure Function Event Grid Subscription #3349

Open amiramit opened 6 years ago

amiramit commented 6 years ago

Presently using ARM I can automate the creation of an Azure Function Subscriber that has a HTTP trigger.

"properties": { "destination": { "endpointType": "WebHook", "properties": { "endpointUrl": "[listsecrets(resourceId('Microsoft.Web/sites/functions', parameters('azurefunctions_name'), parameters('azurefunctions_loadTcsItemSubscription')),'2015-08-01').trigger_url]" } }, "filter": { "includedEventTypes": [ "All" ] } },

I am however unable to do the same for an Azure Function Event Grid trigger, there seems to be no way of accessing the system key.

Many Thanks

robdmoore commented 6 years ago

I've managed to get this working with the following code:


function Get-AccessToken($tenantId) {
    $context = Get-AzureRmContext
    $cache = $context.TokenCache
    $cacheItem = $null
    try {
        $cacheItem = $cache.ReadItems() | Where-Object { $_.TenantId -eq $tenantId } | Select-Object -First 1
    } catch {
        $cache = [Microsoft.IdentityModel.Clients.ActiveDirectory.TokenCache]::DefaultShared
    }
    if (-not $cacheItem) {
        # Service principals don't have a tenant id specified in the cache item
        $cacheItem = $cache.ReadItems() | Select-Object -First 1
        if ($cacheItem.TenantId) {
            throw "Couldn't find an access token for the current user"
        }
    }
    return $cacheItem.AccessToken
}

  Write-Host "Getting master key for $functionName function app"
  $headers = @{
    Authorization = "Bearer $(Get-AccessToken -tenantId $TenantId)"
  }
  $masterKeyContent = Invoke-WebRequest -Uri "https://$functionName.scm.azurewebsites.net/api/functions/admin/masterkey" -Headers $headers | select -Expand Content | ConvertFrom-Json
  $masterKey = $masterKeyContent.masterKey.ToString()

  Write-Host "Getting system key for the Event Grid extension"
  $attempt = 1
  $content = $null
  while ((-not $content) -and $attempt -le 12) {
    if ($attempt -gt 1) {
        Start-Sleep -Seconds 10
    }

    try {
        $content = Invoke-WebRequest "https://$functionName.azurewebsites.net/admin/host/systemkeys/eventgridextensionconfig_extension?code=$masterKey" | select -Expand Content | ConvertFrom-Json
    } catch {
        Write-Warning "Error getting key, trying again"
        Write-Warning $_
    }
    $attempt = $attempt + 1
  }
  $eventGridWebHookCode = $content.value.ToString()

Then pass in the $eventGridWebHookCode to your ARM script as a parameter from PowerShell and you can use it similar to the example below (just change SurveyEmailer with your function name and Swap out SurveyReady with All, or a list of the types you want to subscribe to - you can also add the prefix etc. filters too) to set up a subscription:

{
            "name": "[concat(parameters('appName'), '/Microsoft.EventGrid/SurveyEmailer')]",
            "type": "Microsoft.EventGrid/topics/providers/eventSubscriptions",
            "location": "[resourceGroup().location]",
            "apiVersion": "2018-01-01",
            "properties": {
                "destination": {
                    "endpointType": "WebHook",
                    "properties": {
                        "endpointUrl": "[concat('https://', reference(resourceId('Microsoft.Web/sites', variables('functionsAppName')), '2015-08-01').defaultHostName, '/runtime/webhooks/eventgrid?functionName=SurveyEmailer&code=', uriComponent(parameters('eventGridWebhookCode')))]"
                    }
                },
                "filter": {
                    "includedEventTypes": [
                        "SurveyReady"
                    ]
                }
            },
            "condition": "[variables('addEventGridSubscriptions')]", 
            "dependsOn": [
                "[resourceId('Microsoft.EventGrid/topics', parameters('appName'))]",
                "[resourceId('Microsoft.Web/sites/', variables('functionsAppName'))]"
            ]
        },

Note, the URL to use for the webhook endpoint is different in the latest v2 functions runtime vs v1 so be sure to get the right one for your code.

There is an issue being tracked here to make this easier: https://github.com/Azure/Azure-Functions/issues/516#issuecomment-417427044.

Note: there is a chicken and egg in that the function app needs to exist with the event grid function extension installed and deployed for the key to be able to be generated using the above code so you either need to execute it as a second ARM template or use a condition (like I have above) and execute the ARM template twice the first time you deploy the function app).

nhart12 commented 5 years ago

@robdmoore 's solution works for me. Just had to change the second request from "https://$functionName.azurewebsites.net/admin/host/systemkeys/eventgridextensionconfig_extension?code=$masterKey" to "https://$functionName.azurewebsites.net/admin/host/systemkeys/eventgrid_extension?code=$masterKey"

and also needed to change Environment variable AzureWebJobsSecretStorageType value to 'Files'

vladislav-mitev commented 5 years ago

@robdmoore, is the template deployment for the subscription still working for you? What you posted is what I can also find in the official examples and documentation, but it does not work for me. It fails at validation stage with the following error: The template resource 'Microsoft.EventGrid/topics/myTopicName/providers/Microsoft.EventGrid/eventSubscriptions/mySubscriptionName' cannot reference itself.

With the exception of some "properties" my template is the same as yours. I have a Topic and a Function app already created. 'mySubscriptionName' is the same as the name of my function app.

blueelvis commented 5 years ago

@robdmoore - This doesn't work with the latest version of Azure Functions because of this.

Configuring Event Grid Subscription with the Azure Function is a pain right now. Also, it seems that only master key works. Other host keys don't even work -_-. Any idea when this can be fixed?

pragnagopa commented 5 years ago

We recently added ARM APIs. @mathewc / @mattchenderson - Do we have documentation on how to automate deploying eventgrid trigger via ARM?

mslot commented 5 years ago

@pragnagopa do you know where there exists more info about this? I need to automate it :)

pragnagopa commented 5 years ago

@mathewc / @mattchenderson - do you have documentation for deploying eventgrid trigger via ARM?

mathewc commented 4 years ago

See related issue https://github.com/Azure/Azure-Functions/issues/1004 which details a related blocking issue. That related race condition issue has been resolved, and the runtime fix is going out in the next Functions release.

mslot commented 4 years ago

@mathewc thanks for the info. Does there exist any doc about getting the master key and system key, so we can retrieve the endpoint to which we want the subscription? Or do you first have to resolve the blocking issue, before doc is written?

alex-nanai commented 4 years ago

Docs pleeeeease 😄

mslot commented 4 years ago

Any update on if the docs has been updated with master key and system key retrieval through ARM?

mathewc commented 4 years ago

Yes - the API doc for host keys is here: Web Apps - List Host Keys. That API returns both master key and system keys. Also, @jcbrooks92 has published a sample template here which shows how to use these APIs to create an EventGrid subscription.

mslot commented 4 years ago

Wow @mathewc!! I need to look into this! Thanks for the magnificent support here on the issue, and the clarification throughout. I really like it :)