The portal is not able to confirm the status of your Key Vault reference at this time. Linux App Service Plan.

thanos1983 commented 2 years ago

Azure Keyvault secret reference not resolving within Azure Linux Function connection string.

We are using this terraform module: azurerm_linux_function_app

When we try to add a reference to a key in the vault the following error appears:

"The portal is not able to confirm the status of your Key Vault reference at this time. Please confirm the status directly from the app by checking whether the environment variables have resolved. Click here to access the app."

I have tried to move the function to the same Resource Group as the keyvault it do not make any difference.

I destroyed the Linux Function and replaced it with azurerm_windows_function_app and it worked as expected.

An alternative but not wanted solution that worked is that I changed the App Service Plan from Dynamic to a Shared plan and it works. It does not work with Dynamic plan on Linux OS.

The same problem was reported here: Azure Keyvault secret reference not resolving within Azure Function connection string and it was closed but it seems the problem persists.

There is a recommended solution key vault reference should now work in Linux consumption to use the new syntax as documented in the docs: Use Key Vault references for App Service and Azure Functions/Reference syntax. I tried both syntaxes both returned the same error on Dynamic plan Y1.

Function works perfectly fine on Functions Premium plan as well.

Let me know if you require any further information on this issue.

CodeAkshayJoy commented 2 years ago

why resoolving keyvault reference needs to be part of Hosting Plan, its just injecting secrets as value ENV variable

Ved2806 commented 2 years ago

Hi @mattchenderson Could you please look into this issue?

Ved2806 commented 2 years ago

Hi @pragnagopa Do you have any inputs on this?

thanos1983 commented 2 years ago

Hi @pragnagopa Do you have any inputs on this?

Hello @Ved2806 ,

Is there anyone who can pick up this task? It affects our migration from Windows Functions to Linux.

Is there something that we can help with?

vinsten99 commented 2 years ago

I'm facing the same issue as well "The portal is not able to confirm the status of your Key Vault reference at this time. Linux App Service Plan with Python". Deployed the resource via Bicep. When i had tried the same with dot net and windows it worked fine but when using a linux and python combo the keyvault reference does not resolve. Is this a bug ?

RockyMM commented 2 years ago

In my case it seems that KeyVault references are not resolved.

vinsten99 commented 2 years ago

A workaround for it is to deploy the fucntionapp with a dedicated app plan or premium, the references get resolved the. Although i did deploy with the dedicated app plan the references did get resolved but now the function in my app gets an error during deployment stating internal error hostruntime not reachable :(

Been using this as ref. https://learn.microsoft.com/en-us/azure/azure-functions/functions-infrastructure-as-code?tabs=bicep

warleyc2009 commented 2 years ago

I had this same problem.

I reseted configurations in Key Vault removing and adding again [Get, List] permissions at [Access Policies] options.

After this, its started to work.

RockyMM commented 2 years ago

@warleyc2009 thanks for the hint. Basically I had the same problem.

frbar commented 2 years ago

Same issue here. Switching to Windows, still on Consumption, "solves" the problem.

No luck with reseting configuration, removing/add permissions, identities, etc...

thanos1983 commented 1 year ago

Same issue here. Switching to Windows, still on Consumption, "solves" the problem.

No luck with reseting configuration, removing/add permissions, identities, etc...

Same for me. For the moment we have switched to Windows but I was hopping that this ticket will be resolved at some point.

Ved2806 commented 1 year ago

Hi @mattchenderson Could you please check with this?

mattchenderson commented 1 year ago

Apologies I missed this before- just to confirm, is it purely an issue with status information? Are values resolving at runtime as expected? I see a few folks mentioning failed resolution, but the core error presented in the original statement seems related to status reporting.

If anyone can help me with a correlation ID, we'll dig in a bit further. https://github.com/Azure/azure-functions-host/wiki/Sharing-Your-Function-App-name-privately

frbar commented 1 year ago

Hi @mattchenderson - Unfortunately it is not only a status information issue: values are not resolved at runtime as expected.

See here: https://github.com/frbar/azure-repro-2248

This only deploys the infrastructure, not code for the function, but is enough to reproduce the issue.

mattchenderson commented 1 year ago

Quick version:

We believe the issue to be scoped to status; Linux Consumption apps are still able to resolve secrets.
We have a fix for the status issue checked in and pending deployment. It will unfortunately not come super quickly.
We will continue looking into other solutions that can come faster than that deployment.

Status information

We are still confirming, but we believe we have identified the cause of the status issue. We already had a fix coded, and that is awaiting deployment. Unfortunately, that deployment window may be long given the layer the issue sits at. We will be looking to see if we can provide alternatives as well during the interim period.

The high-level summary is that there was a regression which impacted the main status reporting path for all SKUs. However, we had a fallback in place that all SKUs other than Linux Consumption were able to leverage. Linux Consumption uniquely does not have that fallback available to it, and that is why you are seeing the difference.

This issue will remain open through deployment of that fix. We do not presently have an ETA and will provide an update once there is line of sight. We will also provide additional updates if any alternative options become available.

Secret resolution

We have confirmed that Linux Consumption apps can still resolve secrets. The status issue may be hiding other issues with configuration, access, etc. If you are seeing issues with resolution, please carefully review your configuration. Unfortunately, it is hard to provide a targeted place to look without the status information providing a hint, but oftentimes the issue is rooted in permissions, network access, or specification of the vault itself. Some folks in this thread mentioned trying the same configuration on another plan type, and that's an option for helping in finding that hint, though differences in how you create the setup could impact success there. If you are unable to identify the cause, a support ticket is likely one of the fastest paths to finding what's wrong.

For the repro example in the previous comment

I think that example has some sequencing issues in that the app will get created with the reference to a secret which does not yet exist. The 404 state would persist until the system attempts to resolve the value again, either on a schedule or due to a configuration change that forces a site restart. The problem is that there is no dependency from the app resource on the secret. Now, the secret depends on the parent vault, which depends on the site (for its identity), so there's a bit of a problem there. A user-assigned identity would allow you to pre-create it, map in the permissions, and then set up the site. You could alternatively move the app settings out of siteConfig and into the config/appsettings subresource of the site, which can have a separate, explicit dependency on the secret (explicit through declaration in Bicep or by getting the URL from the secret resource).

ghost commented 1 year ago

This issue has been automatically marked as stale because it has been marked as requiring author feedback but has not had any activity for 4 days. It will be closed if no further activity occurs within 3 days of this comment.

thanos1983 commented 1 year ago

Do not close.

ryan-suarez commented 1 year ago

Please do not close. I am having this exact issue with Azure Function, python 3.9 on linux. The status for Keyvault Reference Details shows as:

"The portal is not able to confirm the status of your Key Vault reference at this time. Please confirm the status directly from the app by checking whether the environment variables have resolved. "

My python code cannot resolve the secret. The secret value is the literal "@Microsoft.Keyvault(SecretUri={myURI})

import os
import logging
import azure.functions as func

## Get Creds
mySecret = os.getenv('mySecret')

def main(mytimer: func.TimerRequest) -> None:
    logging.info('mySecret ' + mySecret)

frbar commented 1 year ago

Following @mattchenderson advises, I've updated the Bicep template to have proper dependencies between function, identity, KV, secrets. Portal is still showing the (i), but values are resolved (tested with a JavaScript function), under Linux and Windows, with System-Assigned or User Managed Identities.

https://github.com/frbar/azure-repro-2248 https://github.com/frbar/azure-repro-2248-system-identity

frbar commented 1 year ago

@ryan-suarez - Indeed. Looks like it depends on the Reference Syntax.

With SecretUri syntax, value is not resolved. Only tested with Linux + System-Assigned identity.
With VaultName + SecretName, it is.

ryan-suarez commented 1 year ago

@frbar - oh, confirmed. VaultName+SecretName does resolve for me.

SecretUri does not resolve with Consumption Plan, but resolves with Premium Plan.

hmarquez-solutions commented 1 year ago

I had an issue where I put a comma instead of a semi-colon in the shorthand VaultName+Secret reference syntax: @Microsoft.KeyVault(VaultName=myvault;SecretName=mysecret)

I was able to fix it but the issue where the portal cannot resolve the reference is still happening for us.

networthdata commented 1 year ago

We have several linux function apps, and we are seeing the same issue as thanos1983.

In some of our apps, key vault references that used to resolve correctly no longer do so.

redhatva commented 1 year ago

We have several linux function apps, and we are seeing the same issue as thanos1983.

In some of our apps, key vault references that used to resolve correctly no longer do so.

Crazy me too... I had a Function App running - Linux / Typescript - Consumption plan... I went and deployed a change via Azure DevOps Pipeline and now my KV references are not evaluating in the function. I could care less about the status - the function needs it more than I do

BeigeBadger commented 1 year ago

Any update on whether this issue is comprehensively solved? I've run into this issue with a Linux Function App on a Consumption plan. Since I'm using Python, I don't have the option to select a Windows host.

I've tried:

Both the full URI syntax, and the Vault Name + Secret Name syntax
Deleting and recreating the Key Vault
Removing and re-adding the MSI for the Function App as a Key Vault Secrets User (including fully removing the MSI and re-enabling it)
Removing and re-adding secrets
Restarting the Function App

The issue is manifesting itself through the blue (i) icon and a message like Thanos originally posted:

"The portal is not able to confirm the status of your Key Vault reference at this time. Please confirm the status directly from the app by checking whether the environment variables have resolved.Click here to access the app."

groovyz commented 1 year ago

I also have a similar problem to @BeigeBadger. In my case, I am not getting an error message ( I only get it when I set it to Slot Deployment) - the Key Vault reference is not resolving to the secret value. I am also writing a Python Azure Function on Linux

uyen-luu commented 10 months ago

Facing same issue with Keyvault reference on Linux function app Consumption plan - Dynamic Y1 The value is still resolved in function app settings, but the portal said, "The portal is not able to confirm the status of your Key Vault reference at this time.". The function still works as all keyvault values still be resolved, but it created deployment issue in DevOps pipeline, WEBSITE_RUN_FROM_PACKAGE Url cannot be resolve due to it cannot access to the storage account which contains the pacakge in the url. It works if I set a pure connection string to AzureWebJobsStorage, so it seems that there is an issue of resolving keyvault value during deployment. Same setting would work Window function app Consumption plan - Dynamic Y1 Any updates on this issue?

bhoogeveen229 commented 8 months ago

I can only mention that I experience the same issue. My App Service Plan runs on Windows. Sometimes the key vault references seems to work, and sometimes they don't and you get the message as @uyen-luu shows: "The portal is not able to confirm the status of your Key Vault reference at this time".

I noticed it mostly happens when I'm using a deployment slot. But, I also saw it happening in the production slot, so it isn't limited to non-production slots unfortunately...

Azure / Azure-Functions