Closed patst closed 9 months ago
Hi @patst, we couldn't reproduce the error using a UserAssignedMSI bot deployed in an Azure App Service. We are working on deploying the bot to an AKS cluster, and it would be helpful if you could provide the steps you followed to deploy your bot and configure the Azure Workload Identity. Thanks!
Hi @patst, we couldn't reproduce the error using a UserAssignedMSI bot deployed in an Azure App Service. We are working on deploying the bot to an AKS cluster, and it would be helpful if you could provide the steps you followed to deploy your bot and configure the Azure Workload Identity. Thanks!
@ceciliaavila thanks for your message.
I created a little example app to reproduce the error. See the repository at https://github.com/patst/botbuilder-js-4582
I added some kubernetes manifests in the manifests
folder.
The configuration in the Azure Portal follows the docs provided on the AKS pages (https://learn.microsoft.com/en-us/azure/aks/workload-identity-overview?tabs=dotnet#microsoft-authentication-library-msal )
Hope that helps
I think the main difference is the ManagedIdentity Credentials (used in the AppService) call the IMDB endpoint at 169.254.169.254
which somehow accepts the scope (or uses another?) and the WorkloadIdentityCredentials use https://login.microsoftonline.com
which rejects the invalid scope
I @patst, thanks for the information. We managed to deploy the application in the cluster and enable workload identity, but we are struggling to create the ingress and the service to access the bot. Do you have the steps or the manifests for this? We are following these two guides, but we are not sure if we are missing something. https://learn.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli https://learn.microsoft.com/en-us/azure/aks/ingress-tls?tabs=azure-cli#create-an-ingress-controller Thanks!
I @patst, thanks for the information. We managed to deploy the application in the cluster and enable workload identity, but we are struggling to create the ingress and the service to access the bot. Do you have the steps or the manifests for this? We are following these two guides, but we are not sure if we are missing something. https://learn.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli https://learn.microsoft.com/en-us/azure/aks/ingress-tls?tabs=azure-cli#create-an-ingress-controller Thanks!
hey @ceciliaavila , thanks for working on it. I added a ingress and service definition to the example repository.
I addition to that, you will need a valid TLS certificate for the ingress. You could use certmanager for that. What problems are you facing exactly? Maybe the AKS team can give you a hand on getting the cluster up and running.
I @patst, thanks for the information. We managed to deploy the application in the cluster and enable workload identity, but we are struggling to create the ingress and the service to access the bot. Do you have the steps or the manifests for this? We are following these two guides, but we are not sure if we are missing something. https://learn.microsoft.com/en-us/azure/aks/ingress-basic?tabs=azure-cli https://learn.microsoft.com/en-us/azure/aks/ingress-tls?tabs=azure-cli#create-an-ingress-controller Thanks!
hey @ceciliaavila , thanks for working on it. I added a ingress and service definition to the example repository.
I addition to that, you will need a valid TLS certificate for the ingress. You could use certmanager for that. What problems are you facing exactly? Maybe the AKS team can give you a hand on getting the cluster up and running.
Hi @patst, thanks for all your help, we were finally able to reproduce the error. We'll be reviewing the fix you proposed. Thanks!
Hi! š
Firstly, thanks for your work on this project! š
Today I used patch-package to patch
botframework-connector@4.21.3
for the project I'm working on.I am using the botbuilder with the msteams connector. My configuration uses a UserAssignedMSI and the botframeworkauthentication is configured like this:
I use it in conjunction with Azure Workload Identity and my bot is running inside a Pod deployed in AKS.
I get an error when a response should be returned to the bot service: (while fetching the token)
The correct scope would be
https://api.botframework.com/.default
I spend some debugging and found a diff comparing the UserAssignedMSI vs the SingleTenant code branch. Single Tenant version: https://github.com/microsoft/botbuilder-js/blob/f3db3e98bb139c7aecc921483ea188574de7aada/libraries/botbuilder-core/src/configurationServiceClientCredentialFactory.ts#L97-L130
If you drill further down it is clear that the
audience
is taken as input and then used asscope
in the oauth flows. This only works, if/.default
is appended to the scope. For the single tenant version this is done in themsalAppCredentials
class: https://github.com/microsoft/botbuilder-js/blob/f3db3e98bb139c7aecc921483ea188574de7aada/libraries/botframework-connector/src/auth/msalAppCredentials.ts#L108-L112In the UserAssignedMSI version the scope is taken without any further modification: https://github.com/microsoft/botbuilder-js/blob/f3db3e98bb139c7aecc921483ea188574de7aada/libraries/botframework-connector/src/auth/managedIdentityAppCredentials.ts#L37
In order to use the same logic like in the SingleTenant version here is the diff that solved my problem:
This issue body was partially generated by patch-package.
What do you think? If you agree I can prepare an pull request for the change