search

Azure / azure-functions-python-worker

Python worker for Azure Functions.
http://aka.ms/azurefunctions
MIT License
335 stars 103 forks source link

BUG In Azure Function Trigger with Managed Identity #1572

Open MiguelElGallo opened 3 weeks ago

MiguelElGallo commented 3 weeks ago

Expected Behavior

If you want to use managed identity for a function to have a trigger binding for event hub you can do:

Python

bp = func.Blueprint()
consumergroup = "$Default"
@bp.event_hub_message_trigger(
    arg_name="event",
    event_hub_name="name",   
    connection="ehconnection",   
    consumer_group=consumergroup)

local.settings.json (for running local) 

localsettings.json(for running locally)
  "ehconnection__fullyQualifiedNamespace": "<namespace>.servicebus.windows.net",
 "ehconnection__credential": "managedIdentity",
 "ehconnection__clientId": "<clientId>"`

This does not work

Actual Behavior

It tries to connect to 169.254.169.254 which is a know address for Azure VMs. But it seems the Azure Function (local) runtime doest no provide that. (see log below)

Most probably in the Azure runtime it will work, but locally it does not work.

Note: In "ehconnection__clientId": "" I have set my user id. I obtained it via : az ad signed-in-user show and field id from there.

[2024-09-03T10:17:58.894Z] Request [e98db5b7-e2aa-486e-be42-6180b172abbb] exception Azure.RequestFailedException: Operation timed out (169.254.169.254:80)
[2024-09-03T10:17:58.895Z]  ---> System.Net.Http.HttpRequestException: Operation timed out (169.254.169.254:80)
[2024-09-03T10:17:58.895Z]  ---> System.Net.Sockets.SocketException (60): Operation timed out
[2024-09-03T10:17:58.895Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ThrowException(SocketError error, CancellationToken cancellationToken)
[2024-09-03T10:17:58.895Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.System.Threading.Tasks.Sources.IValueTaskSource.GetResult(Int16 token)
[2024-09-03T10:17:58.895Z]    at System.Net.Sockets.Socket.<ConnectAsync>g__WaitForConnectWithCancellation|277_0(AwaitableSocketAsyncEventArgs saea, ValueTask connectTask, CancellationToken cancellationToken)
[2024-09-03T10:17:58.895Z]    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    --- End of inner exception stack trace ---
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
[2024-09-03T10:17:58.896Z]    at System.Threading.Tasks.TaskCompletionSourceWithCancellation`1.WaitWithCancellationAsync(CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.GetHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T10:17:58.896Z]    at System.Net.Http.HttpConnectionPool.SendWithVersionDetectionAndRetryAsync(HttpRequestMessage request, Boolean async, Boolean doRequestAuth, CancellationToken cancellationToken)
[2024-09-03T10:17:58.897Z]    at System.Net.Http.HttpClient.<SendAsync>g__Core|83_0(HttpRequestMessage request, HttpCompletionOption completionOption, CancellationTokenSource cts, Boolean disposeCts, CancellationTokenSource pendingRequestsCts, CancellationToken originalCancellationToken)
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
[2024-09-03T10:17:58.897Z]    --- End of inner exception stack trace ---
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.HttpClientTransport.ProcessAsync(HttpMessage message, Boolean async)
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.HttpPipelineTransportPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline)
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.ResponseBodyPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-09-03T10:17:58.897Z]    at Azure.Core.Pipeline.LoggingPolicy.ProcessAsync(HttpMessage message, ReadOnlyMemory`1 pipeline, Boolean async)
[2024-09-03T10:18:05.586Z] Request [e98db5b7-e2aa-486e-be42-6180b172abbb] attempt number 4 took 75.0s
[2024-09-03T10:18:05.586Z] Request [e98db5b7-e2aa-486e-be42-6180b172abbb] GET http://169.254.169.254/metadata/identity/oauth2/token?api-version=2018-02-01&resource=REDACTED&client_id=REDACTED

Steps to Reproduce

1 Write an azure function that has Azure EventHub inbound binding 2 configure it to use managed identity 3 try to run it locally

Relevant code being tried

No response

Relevant log output

No response

requirements.txt file

No response

Where are you facing this problem?

Local - Core Tools

Additional Information

No response

MiguelElGallo commented 3 weeks ago

Update:

If I remove "ehconnection__credential": "managedIdentity", "ehconnection__clientId": "<clientId>"

Then I will get the following error when running locally, but then it will retry automatically , and it will work.

It seems some processes are faster than others.

 [2024-09-03T11:38:31.361Z] False MSAL 4.60.3.0 MSAL.CoreCLR .NET 6.0.31 MacOS [2024-09-03 11:38:31Z - b794dc2f-386f-49ff-a11e-fa4388920033] Exception type: Azure.Identity.CredentialUnavailableException
[2024-09-03T11:38:31.361Z] ---> Inner Exception Details
[2024-09-03T11:38:31.361Z] Exception type: Azure.RequestFailedException
[2024-09-03T11:38:31.361Z] Date:Tue, 03 Sep 2024 11:38:31 GMT
[2024-09-03T11:38:31.361Z] ---> Inner Exception Details
[2024-09-03T11:38:31.361Z] Connection:keep-alive
[2024-09-03T11:38:31.362Z] Exception type: System.Net.Http.HttpRequestException
[2024-09-03T11:38:31.362Z] Keep-Alive:REDACTED
[2024-09-03T11:38:31.362Z] 
[2024-09-03T11:38:31.362Z] ---> Inner Exception Details
[2024-09-03T11:38:31.362Z] Exception type: System.Net.Sockets.SocketException
[2024-09-03T11:38:31.362Z] To see full exception details, enable PII Logging. See https://aka.ms/msal-net-logging
[2024-09-03T11:38:31.362Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.CreateException(SocketError error, Boolean forAsyncThrow)
[2024-09-03T11:38:31.362Z]    at System.Net.Sockets.Socket.AwaitableSocketAsyncEventArgs.ConnectAsync(Socket socket)
[2024-09-03T11:38:31.362Z]    at System.Net.Sockets.Socket.ConnectAsync(EndPoint remoteEP, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.ConnectToTcpHostAsync(String host, Int32 port, HttpRequestMessage initialRequest, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.ConnectAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.362Z]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
[2024-09-03T11:38:31.363Z]    at System.Net.Http.HttpConnectionPool.CreateHttp11ConnectionAsync(HttpRequestMessage request, Boolean async, CancellationToken cancellationToken)
[2024-09-03T11:38:31.363Z]    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
[2024-09-03T11:38:31.363Z]    at System.Runtime.CompilerServices.AsyncMethodBuilderCore.Start[TStateMachine](TStateMachine& stateMachine)
[2024-09-03T11:38:31.363Z]    at System.Net.Http.HttpConnectionPool.AddHttp11ConnectionAsync(HttpRequestMessage request)
[2024-09-03T11:38:31.363Z]    at System.Net.Http.HttpConnectionPool.<>c__DisplayClass75_0.<CheckForHttp11ConnectionInjection>b__0()
[2024-09-03T11:38:31.363Z]    at System.Threading.Tasks.Task`1.InnerInvoke()
[2024-09-03T11:38:31.363Z]    at System.Threading.Tasks.Task.<>c.<.cctor>b__272_0(Object obj)