Azure function to SignalR using MI fails with 401 - "The signature key was not found"

komjera commented 5 months ago

Describe the bug

When makings calls to a signalR service from a Function App we very intermittently get 401 responses, not on every request but on large portion of the requests. Before and after any given 401 there are succesful requests from the same function app. This has happened twice over that last 2 months, that last time we had the issue it wasn't resolved until we restarted the SignalR service.

To Reproduce

We havent been able to reproduce the issue, we have only encountered it twice. The first time we didn't do anything and it was working in a couple of hours. The second time the issue persisted multiple hours until we restarted the SignalR Service.

Exceptions (if any)

From application insights logs

Microsoft.Azure.WebJobs.Host.FunctionInvocationException: Exception while executing function: Functions.PushQrSigningCodeAndHint
---> Microsoft.Azure.SignalR.Common.AzureSignalRUnauthorizedException: Authorization failed. If you were using AccessKey, please check connection string and see if the AccessKey is correct. If you were using Azure Active Directory, please note that the role assignments will take up to 30 minutes to take effect if it was added recently. Request Uri: https://{serviceName}.service.signalr.net/api/hubs/{hubName}/users/{userId}/:send?api-version=2022-06-01
---> System.Net.Http.HttpRequestException: Response status code does not indicate success: 401 (Unauthorized)
--- End of inner exception stack trace ---

Capturing trace logs we were able to find more details (A bearer token from last failure is saved if any information contained in it is needed)

Further technical details

Microsoft.Azure.Functions.Worker.Extensions.SignalRService 1.13.2
We are using Managed Identity to authenticate our Azure Function with SignalR
azurefunctions sdkVersion 4.34.1.22669
Function runtime is dotnet 8 isolated on windows

terencefan commented 4 months ago

Could you share us with your ResourceId and precise timestamp so we could take a look about this issue?

komjera commented 3 months ago

Sorry for delaying the issue.

Is the ResourceId safe to share in public or do you want me to send it to you some other way?

Timestamp is 2024-05-27 14:48:46 GMT.

terencefan commented 3 months ago

Does the problem still exist? There was a known issue that will lead to 401 exceptions and has been fixed ~1 month ago.

Azure / azure-signalr