pulumi / pulumi-azure

A Microsoft Azure Pulumi resource package, providing multi-language access to Azure
Apache License 2.0
133 stars 50 forks source link

Notes on bringing up a Functions example in Azure #6

Closed lukehoban closed 6 years ago

lukehoban commented 6 years ago

Not a bug per se - just a story. (though the story ends with it being seemingly impossible to use Azure Functions from Terraform/Pulumi - which we certainly will want to figure out how to fix).

Azure docs tell us how to stand up Azure Functions: https://docs.microsoft.com/en-us/azure/azure-functions/functions-infrastructure-as-code.

Starting off is simple:

import * as azure from "@pulumi/azurerm";

let name = "serverlessraw"

let resourceGroup = new azure.core.ResourceGroup(name, {
    location:"westus",
});

let storageAccount = new azure.storage.Account(name, {
    resourceGroupName: resourceGroup.name,
    location: resourceGroup.location,
    accountTier: "Standard",
    accountReplicationType: "LRS",
});

This works like a charm!

Next we need what Azure docs call a HostingPlan (though it's name is actually serverFarm). We find a Plan type that looks right. Azure docs make it sound like we should pass magic constant "Dynamic" as SKU properties on this Plan. The names for TF are totally different than for AzureRM (and the names for the Azure SDK are different than either of these). All with unclear mappings between them (computeMode vs. name vs. size - which I think are actually all the same thing??).

$ yarn build && pulumi up
yarn run v1.3.2
$ tsc
✨  Done in 1.62s.
Performing changes:
* pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:local::serverless-raw::pulumi:pulumi:Stack::serverless-raw-local]
    + azurerm:appservice/plan:Plan: (create)
        [urn=urn:pulumi:local::serverless-raw::azurerm:appservice/plan:Plan::serverlessraw]
        kind             : "Windows"
        location         : "westus"
        name             : "serverlessraw2afa398e"
        resourceGroupName: "serverlessrawade85cd9"
        sku              : [
            [0]: {
                size: "Dynamic"
                tier: "Dynamic"
            }
        ]
error PU2003: Plan apply failed: rpc error: code = Unknown desc = web.AppServicePlansClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="The parameter SKU.Name has an invalid value." Details=[{"Message":"The parameter SKU.Name has an invalid value."},{"Code":"BadRequest"},{"ErrorEntity":{"Code":"BadRequest","Message":"The parameter SKU.Name has an invalid value.","MessageTemplate":"The parameter {0} has an invalid value.","Parameters":["SKU.Name"]}}]
Step #4 failed [create]: this failure was catastrophic and the provider cannot guarantee recovery
info: no changes required:
      3 resources unchanged
A catastrophic error occurred; resources states may be unknown
error: rpc error: code = Unknown desc = web.AppServicePlansClient#CreateOrUpdate: Failure responding to request: StatusCode=400 -- Original Error: autorest/azure: Service returned an error. Status=400 Code="BadRequest" Message="The parameter SKU.Name has an invalid value." Details=[{"Message":"The parameter SKU.Name has an invalid value."},{"Code":"BadRequest"},{"ErrorEntity":{"Code":"BadRequest","Message":"The parameter SKU.Name has an invalid value.","MessageTemplate":"The parameter {0} has an invalid value.","Parameters":["SKU.Name"]}}]

Okay - that didn't work. Googling we find https://github.com/terraform-providers/terraform-provider-azurerm/issues/131#issuecomment-326539161 which suggests we use Y1 instead of Dynamic. I'm serious. Y1. No explanation - just Y1.

$ yarn build && pulumi up
yarn run v1.3.2
$ tsc
✨  Done in 1.59s.
Performing changes:
* pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:local::serverless-raw::pulumi:pulumi:Stack::serverless-raw-local]
    + azurerm:appservice/plan:Plan: (create)
        [urn=urn:pulumi:local::serverless-raw::azurerm:appservice/plan:Plan::serverlessraw]
        kind             : "Windows"
        location         : "westus"
        name             : "serverlessraw0c900d59"
        resourceGroupName: "serverlessrawade85cd9"
        sku              : [
            [0]: {
                size: "Y1"
                tier: "Dynamic"
            }
        ]
        ---outputs:---
        id                    : "/subscriptions/8484c44e-3967-4ae7-abad-0c187c09e18e/resourceGroups/serverlessrawade85cd9/providers/Microsoft.Web/serverfarms/serverlessraw0c900d59"
        kind                  : "functionapp"
        maximumNumberOfWorkers: "0"
        properties            : [
            [0]: {
                perSiteScaling: false
                reserved      : false
            }
        ]
        sku                   : [
            [0]: {
                capacity: "0"
                size    : "Y1"
                tier    : "Dynamic"
            }
        ]
    + azurerm:appservice/appService:AppService: (create)
        [urn=urn:pulumi:local::serverless-raw::azurerm:appservice/appService:AppService::serverlessraw]
        appServicePlanId : "/subscriptions/8484c44e-3967-4ae7-abad-0c187c09e18e/resourceGroups/serverlessrawade85cd9/providers/Microsoft.Web/serverfarms/serverlessraw0c900d59"
        enabled          : true
        location         : "westus"
        name             : "serverlessraw645adfef"
        resourceGroupName: "serverlessrawade85cd9"
error PU2003: Plan apply failed: rpc error: code = Unknown desc = web.AppsClient#CreateOrUpdate: Failure sending request: StatusCode=409 -- Original Error: failed request: autorest/azure: Service returned an error. Status=<nil> <nil>
Step #5 failed [create]: this failure was catastrophic and the provider cannot guarantee recovery
info: 1 change performed:
    + 1 resource created
      3 resources unchanged
Update duration: 11.912547416s
A catastrophic error occurred; resources states may be unknown
error: rpc error: code = Unknown desc = web.AppsClient#CreateOrUpdate: Failure sending request: StatusCode=409 -- Original Error: failed request: autorest/azure: Service returned an error. Status=<nil> <nil>

And that worked! Except the next step failed.

Any guesses what that error means? Nope. And not a single piece of useful content there to help with googling. I suppose the <nil>s were probably where the useful information was supposed to be :-).

Let's look at TF sources to see if that helps: https://github.com/terraform-providers/terraform-provider-azurerm/blob/master/azurerm/resource_arm_app_service.go#L270.

        // NOTE: these seem like sensible defaults, in lieu of any better documentation.
    skipDNSRegistration := false
    forceDNSRegistration := false
    skipCustomDomainVerification := true
    ttlInSeconds := "60"
    _, createErr := client.CreateOrUpdate(resGroup, name, siteEnvelope, &skipDNSRegistration, &skipCustomDomainVerification, &forceDNSRegistration, ttlInSeconds, make(chan struct{}))
    err := <-createErr
    if err != nil {
        return err
    }

Well - that first section doesn't inspire confidence does it! Nothing obvious there, let's go back to the docs.

Well - the docs told me there were some required appSettings - let's try adding those:

"appSettings": [
    {
        "name": "AzureWebJobsStorage",
        "value": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('storageAccountName'), ';AccountKey=', listKeys(variables('storageAccountid'),'2015-05-01-preview').key1)]"
    },
    {
        "name": "AzureWebJobsDashboard",
        "value": "[concat('DefaultEndpointsProtocol=https;AccountName=', variables('storageAccountName'), ';AccountKey=', listKeys(variables('storageAccountid'),'2015-05-01-preview').key1)]"
    }

Hmm - what on earth is that listKeys thing. Even after reading the docs for 10 mins I can't figure out what the above is actually going to do. But there is a primaryAccessKey on the output properties of the resource we created earlier - let's just try using that.

async function getConnectionString() {
    let name = await storageAccount.name;
    let primaryAccessKey = await storageAccount.primaryAccessKey;
    return `DefaultEndpointsProtocol=https;AccountName=${name};AccountKey=${primaryAccessKey}`
}

let sites = new azure.appservice.AppService(name, {
    resourceGroupName: resourceGroup.name,
    location: resourceGroup.location,
    appServicePlanId: hostingPlan.id,
    appSettings: [{
        AzureWebJobsDashboard: getConnectionString(),
        AzureWebJobsStorage: getConnectionString(),
    }],    
})

Actually kind of nice how Pulumi let's us factor out that crazy connection string construction.

But results in the same error. :-(

Diving into the template linked from the the docs, we see they actually pass a bunch of other settings beyond what's listed in the docs. Let's try adding those:

let sites = new azure.appservice.AppService(name, {
    resourceGroupName: resourceGroup.name,
    location: resourceGroup.location,
    appServicePlanId: hostingPlan.id,
    enabled: true,
    appSettings: [{
        AzureWebJobsDashboard: getConnectionString(),
        AzureWebJobsStorage: getConnectionString(),
        WEBSITE_CONTENTAZUREFILECONNECTIONSTRING: getConnectionString(),
        WEBSITE_CONTENTSHARE: name,
        FUNCTIONS_EXTENSION_VERSION: "~1",
        WEBSITE_NODE_DEFAULT_VERSION: "6.5.0",
    }],

})

And updating:

$ yarn build && pulumi up
yarn run v1.3.2
$ tsc
✨  Done in 1.61s.
Performing changes:
* pulumi:pulumi:Stack: (same)
    [urn=urn:pulumi:local::serverless-raw::pulumi:pulumi:Stack::serverless-raw-local]
info: panic: fatal: An assertion has failed: Unexpected duplicate underscore: f_u_n_c_t_i_o_n_s__e_x_t_e_n_s_i_o_n__v_e_r_s_i_o_n
info:
info: goroutine 39 [running]:
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi/pkg/util/contract.failfast(0xc4200700e0, 0x6d)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi/pkg/util/contract/failfast.go:11 +0xf3
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi/pkg/util/contract.Assertf(0x227c000, 0x25067c2, 0x23, 0xc420a10db8, 0x1, 0x1)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi/pkg/util/contract/assert.go:21 +0x13d
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge.TerraformToPulumiName(0xc4202fc780, 0x33, 0xc4202fc700, 0x33, 0x3009b80)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge/names.go:40 +0x23e
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge.getInfoFromTerraformName(0xc4202fc780, 0x33, 0x0, 0x0, 0xc420236500, 0x4, 0xc420236360, 0x0, 0x0)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge/schema.go:453 +0x11f
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge.MakeTerraformOutputs(0xc420236390, 0x0, 0x0, 0xc420a11000, 0x104722a)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge/schema.go:232 +0x12f
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge.MakeTerraformOutput(0x2302ec0, 0xc420236390, 0x0, 0x0, 0x0, 0xc420663240, 0xb)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge/schema.go:299 +0x623
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge.MakeTerraformOutput(0x22587a0, 0xc420616c80, 0xc420312c30, 0x0, 0xc420377a00, 0xc420663240, 0xb)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge/schema.go:285 +0x177
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge.MakeTerraformOutputs(0xc420236360, 0xc42030e180, 0xc4203496e0, 0xc420236400, 0x0)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge/schema.go:236 +0x1ae
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge.(*Provider).Check(0xc42038a200, 0x2f763e0, 0xc4202361e0, 0xc4206169e0, 0xc42038a200, 0x0, 0x0)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi-terraform/pkg/tfbridge/provider.go:246 +0x73b
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi/sdk/proto/go._ResourceProvider_Check_Handler.func1(0x2f763e0, 0xc4202361e0, 0x23dd780, 0xc4206169e0, 0x2f763e0, 0xc4202361e0, 0x2f7b0e0, 0xc4208e05a0)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi/sdk/proto/go/provider.pb.go:567 +0x86
info: github.com/pulumi/pulumi-azure/vendor/github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc.OpenTracingServerInterceptor.func1(0x2f763e0, 0xc4202361e0, 0x23dd780, 0xc4206169e0, 0xc420616a20, 0xc420616a40, 0x0, 0x0, 0x0, 0x0)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/grpc-ecosystem/grpc-opentracing/go/otgrpc/server.go:61 +0x326
info: github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi/sdk/proto/go._ResourceProvider_Check_Handler(0x2469ba0, 0xc42038a200, 0x2f763e0, 0xc420236060, 0xc42011e070, 0xc420374940, 0x0, 0x0, 0x0, 0x0)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/github.com/pulumi/pulumi/sdk/proto/go/provider.pb.go:569 +0x16d
info: github.com/pulumi/pulumi-azure/vendor/google.golang.org/grpc.(*Server).processUnaryRPC(0xc420001c80, 0x2f79ee0, 0xc420278600, 0xc4208e6b40, 0xc4203732f0, 0x2fdd490, 0x0, 0x0, 0x0)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/google.golang.org/grpc/server.go:900 +0x9d3
info: github.com/pulumi/pulumi-azure/vendor/google.golang.org/grpc.(*Server).handleStream(0xc420001c80, 0x2f79ee0, 0xc420278600, 0xc4208e6b40, 0x0)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/google.golang.org/grpc/server.go:1122 +0x1528
info: github.com/pulumi/pulumi-azure/vendor/google.golang.org/grpc.(*Server).serveStreams.func1.1(0xc4203e8010, 0xc420001c80, 0x2f79ee0, 0xc420278600, 0xc4208e6b40)
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/google.golang.org/grpc/server.go:617 +0x9f
info: created by github.com/pulumi/pulumi-azure/vendor/google.golang.org/grpc.(*Server).serveStreams.func1
info:   /Users/luke/go/src/github.com/pulumi/pulumi-azure/vendor/google.golang.org/grpc/server.go:615 +0xa1
info: no changes required:
      4 resources unchanged
error: rpc error: code = Unavailable desc = transport is closing

f_u_n_c_t_i_o_n_s__e_x_t_e_n_s_i_o_n__v_e_r_s_i_o_n. Well that's fun :-). And this time it's our fault.

Reading the TF impl, it looks like these settings aren't even applied till after the initial Create succeeds, so let's try something else.

The docs mention setting "kind": "function". But there's no kind on the TF resource. And - indeed, the TF provider implementation does not offer any way to pass a value there. Perhaps that's why https://github.com/terraform-providers/terraform-provider-azurerm/issues/131#issuecomment-326539161 is still open :-).

Maybe this really is impossible in current state after all.

So, my current guess is that when you pick Y1 as your plan, it somehow requires you to set kind=function? But reports an unusable error message if you don't do that.

Oh well - maybe next time...

lindydonna commented 6 years ago

Yes, the consumption plan Y1 can only be used with kind functionapp.

Y1 is likely documented in the REST API docs. The problem is probably that the documentation is using a newer version of the REST API then TF.

At any rate you can test creation using a regular app service plan with SKU B1. Even F1 "free" should work.

ListKeys is indeed horrible, that's where the ARM DSL comes in. You can even test that part of the code by using a regular web app.

lukehoban commented 6 years ago

Thanks @lindydonna :+1:

I couldn't actually find any docs on Y1, just mentions of it in discussion threads - do you know where consumption plan options are documented?

Good point on using a non-consumption plan - though we definitely want to support consumption plans for anything we do in @pulumi/cluod.

Agreed that ListKeys appears to be a frequently required ARM DSL function. Provides a nice comparison point for how much nicer this can be in Pulumi.

Sounds like next step will need to be patching the TF azure provider to support passing kind in - and see if that can unblock things (then upstreaming to fix https://github.com/terraform-providers/terraform-provider-azurerm/issues/131#issuecomment-326539161).

lindydonna commented 6 years ago

Just so this is recorded here, I think Y1 was the old SKU name with an older version of the App Service ARM REST API. Terraform is probably using an older version of the API, so only the new one is documented.

In lieu of real reference documentation, this is actually how you create a Function App:

I've followed up on the Azure REST spec feature that was linked from https://github.com/terraform-providers/terraform-provider-azurerm/issues/131 and asked David Ebbo to confirm the approach I suggested.

lindydonna commented 6 years ago

There's an API for SKU options: https://github.com/Azure/azure-rest-api-specs/blob/current/specification/web/resource-manager/Microsoft.Web/2016-09-01/AppServicePlans.json#L790-L825

I gave the App Service team feedback that they should link it more prominently. :)

joeduffy commented 6 years ago

I don't think there's any action item here, so closing out.