Azure / Azure-Functions

1.11k stars 196 forks source link

arm template deployed function app has no host keys and throws errors #1080

Open myusrn opened 5 years ago

myusrn commented 5 years ago

I've been using a resource group, where i have a freshly minted function app | automation script | download to acquire the bash/powershell script story for recreating the environment in another resource group or azure subscription.

What i'm experiencing with these azure resource manager [arm] template driven deployments of function app end up without function app settings | host keys (all functions) | _master & default in place.

When I then attempt to publish a visual studio 2017 functions v2 project into this arm template created function it prompts me about upgrading it to v2 of the runtime which I accept. Afterwards I still don't have function app settings | host keys (all functions) | _master & default host keys in place and start seeing dialogs like those I screen grabbed here .

Perhaps there is some way to define arm template.json settings for azure function app so that it starts out life already configured for v2 runtime and with randomly generated host keys (all functions) | _master & default in place like you get when you use the portal UI.

ColbyTresness commented 5 years ago

@mathewc

myusrn commented 5 years ago

. . . I just retested creating and deploying an azure resource manager [arm] template containing a current Runtime version: 2.0.12438.0 function app definition. As before the arm template created function app in the new resource group was created initially as Runtime version: 1.0.12299.0 function app with valid _master and default host keys in place.

When i then clicked the ~2 setting as shown here, instead of using the vstudio function app project deployment into that new function app setup, to drive the upgrade to 2.0.12438.0, i still get error dialog about host keys shown in this screen grab.

Just wanted to highlight that this function app v2 arm template originated provisioning issue still exists. The work around is to delete and recreate the function app target interactively using the portal.

timtos commented 5 years ago

I have the same or a similar problem.

Function app gets deployed, everything is working fine and I can navigate everywhere in the portal without any error messages but there are no keys available. And I didn't find a way to get them generated... at least not a reproducable.

Any updates on this?

Chedberg84 commented 5 years ago

I am also seeing this issue when using the Azure CLI command for creating a slot.

az functionapp deployment slot create ...

My function app slot does not have any "function keys" or "host keys" defined after this runs. Whenever I click on the "manage" slot in the Azure portal it throws this exception.

Error: Encountered an error (InternalServerError) from host runtime.

I tried adding a default function key and host key manually through the portal and the save command fails, throwing the same error as above.

I noticed when creating a function app through a Terraform resource azurerm_function_app, the keys do show up as expected and the function app doesn't throw these errors. However, in order to build a process using deployment slots on an Azure function application, we have to use the Azure CLI for now which appears to be broken. There is no Terraform "slot" resource available for an Azure function currently.

Chedberg84 commented 5 years ago

Update

I was able to work around this issue by including the following 2 Application Settings:

With those key names included and values pointing at a storage account connection string, I now see the 'default' and 'master' host keys through the Azure portal.

myusrn commented 5 years ago

. . I just retested creating function app in portal > changing it from default 1.0.12615.0 (~1) to 2.0.12653.0 (~2) runtime > selecting app service plan + app service + storage account + app insights [ which doesn't get captured in template ] > export and download the arm template for those portal UI created set of resources. When i then test deleting the resource group and redeploying the contents using the arm template ./deploy.sh script then as before i get a v1 function app with a valid _master and default host keys in place.

When i then change it from a v1 [ 1.0.12615.0 ] function app to a v2 [ 2.0.12653.0 ] runtime setup the _master and default host keys continuing to work. If on the other hand I change it from a v1 [ 1.0.12615.0 ] function app to a v2 [ 2.0.12653.0 ] runtime setup using the vs19 function app deployment process which requires v2 then the _master and default host keys stop working as outlined at the start of this issue.

So it appears i need to set the vs19 function app project deployment remote settings for AzureWebJobsDashboard and AzureWebJobsSettings to be < cloud storage account connection string > in order for vs19 deployment driven upgrade of runtime setting to leave the _master and default host keys in working state after deployment is completed.

note that i have to make the following windows workstation msi installed azure-cli modification in order for git-bash.exe environment executed arm template ./deploy.sh script calls to leverage it: echo "/c/Program Files (x86)/Microsoft SDKs/Azure/CLI2/python.exe" -IBm azure.cli $@ > "%programfiles(X86)%\Microsoft SDKs\Azure\CLI2\wbin\az"

myusrn commented 5 years ago

. . . did some more tests and found this has nothing to do with vs19 function app project deployment upgrade of an existing portal created function app target, i.e. i can upgrade that function app target in portal from v1 to v2 before the vs19 deployment is carried out and result will repro.

I also tested setting just AzureWebJobsDashboard and just AzureWebJobsStorage with valid storage account connection string in lieu of UseDevelopmentStorage=true and only AzureWebJobsStorage needs to be set in default functions app project case for this master and default host keys issue to not arise after deployment.

It appears the only issue here is developer assumption that when publishing a vs19 function app project that there isn't a need to provide a Remote deployment value for AzureWebJobsStorage in the profile settings. This suggests that when you first create a publishing profile in a new project that you need to use the "Create Profile" vs "Publish Immediately" option so you have a chance to go in and apply that setting. Alternatively if the function app target already provisioned using the portal UI has the AzureWebJobsStorage application setting defined then the profile wizard just pulls that in.

Unless there is another scenario driving this issue that isn't addressed by what i outline here then this issue should be closed.

pioardi commented 4 years ago

I hope this will help people to fix that , I spent 2/3 days to find the problem . I was leaving "netFrameworkVersion" to "" for the site config , set this value to v4.0 fixed the problem.

Other possible causes could be related to the storage configuration and firewall .

csuzw commented 4 years ago

I'm seeing this issue now and I cannot work out why. I have another function app that works and I copied the ARM template from it so all the config and settings are the same as far as I can tell (netFrameworkVersion is v4.0 and AzureWebJobsStorage does exist and has the correct value) but I don't have the _master/default host keys. I've tried deleting the function app and recreating and it still doesn't work. FUNCTION_EXTENSION_VERSION is ~3. This is all being deployed (both ARM template and code) via an Azure DevOps pipeline. I'm pretty sure I've encountered this before when I wasn't setting the standard app settings like AzureWebJobsStorage correctly but I definitely am doing them correctly this time (or at least the same as another function app that does work).

pioardi commented 4 years ago

@csuzw your storage account has any vnet restriction on the firewall ?

csuzw commented 4 years ago

It is connected to a vnet but it's an identical setup to another function app that works correctly. However I rebuilt everything again today (which I did twice yesterday), no changes that I'm aware of and it's worked this time so I guess I've missed something somewhere so I guess this is sorted now. I just wish there was proper logging/warning/error messages in this situation.

weskroesbergen commented 4 years ago

I'm seeing this issue now and I cannot work out why. I have another function app that works and I copied the ARM template from it so all the config and settings are the same as far as I can tell (netFrameworkVersion is v4.0 and AzureWebJobsStorage does exist and has the correct value) but I don't have the _master/default host keys. I've tried deleting the function app and recreating and it still doesn't work. FUNCTION_EXTENSION_VERSION is ~3. This is all being deployed (both ARM template and code) via an Azure DevOps pipeline. I'm pretty sure I've encountered this before when I wasn't setting the standard app settings like AzureWebJobsStorage correctly but I definitely am doing them correctly this time (or at least the same as another function app that does work).

I found that toggling the functions version to V2 in the portal and back to V3 seems to fix the app not starting. Suspect some sort of initialization process not getting triggered on ARM deploy.

rory-a-hunt commented 4 years ago

If your host keys fail to appear after you have deleted and redeployed your function app (I had this issue after upgrade from ~2 to ~3) then ensure all file shares and blob-containers (in used storage account) with the function app name are also deleted before trying redeploy. specifically the following containers: azure-webjobs-hosts azure-webjobs-secrets

Only by clearing these was I able to get a clean re-deploy - and host/app keys back again!

ann0nc0d3r commented 3 years ago

I'm still having this problem.

@mathewc - This has been an issue since 2018. Can we please get some sort of response/update as to progress being made? This makes working with Azure Functions practically impossible.

ann0nc0d3r commented 3 years ago

@myusrn - How did you "provide a Remote deployment value for AzureWebJobsStorage in the profile settings"? Which profile settings are you referring to? local.settings.json?