Unable to connect to localhost eventhub emulator using Python Azure Function library

[Goldstorm]

Describe the bug When setting up an in bound or outbound eventhub trigger utilizing the azurefunction library, the function trigger fails to start with this error:

[2024-06-06T13:46:01.578Z] The listener for function 'Functions.inbound_eh' was unable to start.
[2024-06-06T13:46:01.580Z] The listener for function 'Functions.inbound_eh' was unable to start. System.Private.CoreLib: One or more errors occurred. (No connection could be made because the target machine actively refused it.). System.Private.CoreLib: No connection could be made because the target machine actively refused it.
[2024-06-06T13:46:01.582Z] Host started (27811ms)

To Reproduce

1. Open Visual Studio Code, ensure you have the Azure Extensions (You do not need Azurite, we can utilize the one provided by the container), and select the following options:
   1. Create New Azure Function
   2. Model V2
   3. Select your Python Interpreter (Mine is 3.11.8)
   4. EventHub Trigger
   5. Name the trigger (inbound_eh)
   6. Name of the Event Hub
   7. Select a Subscription and EventHub, these wont matter as they will be replaced.
2. Create a Python Azure Function Event Hub example utilizing the one provided here by Microsoft.
3. Create a local.settings.json with the following config:

   {
   "IsEncrypted": false,
   "Values": {
   "AzureWebJobsStorage": "UseDevelopmentStorage=true",
   "FUNCTIONS_WORKER_RUNTIME": "python",
   "AzureWebJobsFeatureFlags": "EnableWorkerIndexing",
   "connection": "Endpoint=sb://localhost;SharedAccessKeyName=RootManageSharedAccessKey;SharedAccessKey=SAS_KEY_VALUE;UseDevelopmentEmulator=true;"
   }
   }

4. Change the function_app.py to reflect this:

   
   import azure.functions as func
   import logging

app = func.FunctionApp()

@app.event_hub_message_trigger(arg_name="azeventhub", event_hub_name="eh-local-topic", connection="connection") def inbound_eh(azeventhub: func.EventHubEvent): logging.info('Python EventHub trigger processed an event: %s', azeventhub.get_body().decode('utf-8'))

5. Ensure your `host.json` is set to `extensionBundle: {"version": "[4.*, 5.0.0)"}`
6. Save, click run and debug.
7. After a few seconds, the listener function wont be able to start due to the error: `System.Private.CoreLib: No connection could be made because the target machine actively refused it.`

**Expected behavior**
I would expect to see Job Host started, and for the function to read from two partitions.

**Screenshots**
If applicable, add screenshots to help explain your problem.

**Desktop:**
 - Windows

**Image Platform-Architecture Used:**
- Linux-amd64

**Docker Version:**
- Podman:

Client:       Podman Engine
Version:      5.0.3
API Version:  5.0.3
Go Version:   go1.22.3
Git Commit:   d08315df35cb6e95f65bf3935f529295c6e54742
Built:        Fri May 10 14:02:40 2024
OS/Arch:      windows/amd64

Server:       Podman Engine
Version:      5.0.3
API Version:  5.0.3
Go Version:   go1.22.2
Built:        Thu May  9 20:00:00 2024
OS/Arch:      linux/amd64

**Arguments && Environment variables to start Emulator:**
- ACCEPT_EULA: "Y"
- CONFIG_PATH: None, hardcoded under volumes.
  - `"D:\\projects\\emulators\\azure-event-hubs-emulator-installer\\Config.json:/Eventhubs_Emulator/ConfigFiles/Config.json"`

**Emulator Launch Method:**
  - podman docker-compose wrapper
    - docker-compose up -d

**Additional context:**
What's very strange is that using the azure-eventhub python package by itself works just fine using this config method. It appears to just be when configuring this through an azure function eventhub trigger.

[Saglodha]

Thanks for reporting this one @Goldstorm. Looking at error it appears to be breaking at network interaction, I have few questions on setup:

1. Are you running functions locally as well?
2. Where are the function and event hubs emulator running? Are those on same machine or separate hosts? How does the network configuration look like?

Can you provide above details on your topology?

[Goldstorm]

Absolutely.

1. Yes, the functions are running locally. They are running through vscode, utilizing a launch.json.
2. Functions are running locally on the same windows PC, and the event hub emulator is running on the same PC via Podman.

Basically, everything is running locally. I am trying to emulate everything without utilizing azure cloud at the moment.

[phs18]

Hi @Goldstorm did you figure out what was the issue even I have same issue

[Saglodha]

Hi @Goldstorm / @phs18 ,

I spent some time repro'ing the issue and realized that it was caused due to older Event Hub extension package bundled with above mentioned extension bundle.

To communicate with emulator, we must use latest extension package. Could you please manually add the below package to solution and retry?

ExtensionPackage="Microsoft.Azure.WebJobs.Extensions.EventHubs" Version="6.3.0"

[phs18]

where do we need to add this @Saglodha

[Goldstorm]

To communicate with emulator, we must use latest extension package. Could you please manually add the below package to solution and retry?

ExtensionPackage="Microsoft.Azure.WebJobs.Extensions.EventHubs" Version="6.3.0"

I am unsure if this is possible within python. This appears to be a nuget package. The latest version for Azure Eventhubs is here in PyPi. However, when loading up the Azure Function, I do see that we are using the Azure 5.5 extension bundle

[2024-06-21T15:00:30.900Z] Loading startup extension 'EventHubs' [2024-06-21T15:00:30.903Z] Loaded extension 'EventHubs' (5.5.0.0)

Where / which file should this be put in if utilizing Python? I am using this guide for the bindings: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs?tabs=isolated-process%2Cextensionv5&pivots=programming-language-python

[Saglodha]

@Goldstorm / @phs18 ,

According to Azure functions documentation. using extension bundles in non .NET language is indeed the recommended approach. Since we recently announced emulator interaction, the underlying extension package isn't updated to reflect latest one in extension bundle.

To get unblocked you can manually add extension package using func extension install command via function core tools. Ensure to remove the extension bundle from host.json before manually installing this package. Below documentation has detailed information to follow:

https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-register#explicitly-install-extensions https://learn.microsoft.com/en-us/azure/azure-functions/functions-core-tools-reference?tabs=v2#func-extensions-install

Command to run:

func extensions install --package Microsoft.Azure.WebJobs.Extensions.EventHubs --version 6.3.0

We realize this isn’t a permanent resolution, but it should help you move forward. We’ll also collaborate internally with functions team to update the extension bundle.

[Goldstorm]

Command to run:

func extensions install --package Microsoft.Azure.WebJobs.Extensions.EventHubs --version 6.3.0

Does not work, unfortunately. I think it can't find it in the particular extension bundle?

I did use 6.3.0, and 6.3.3 fyi

[ranma2913]

Command to run: func extensions install --package Microsoft.Azure.WebJobs.Extensions.EventHubs --version 6.3.0

Does not work, unfortunately. I think it can't find it in the particular extension bundle?

I did use 6.3.0, and 6.3.3 fyi

First remove extensinoBundle from host.json

Then you'll be able to add the package, & it will create an extensions.csproj file like in my PR #20

[phs18]

@ranma2913 DID THIS SETTING WORK FOR YOU IT THRE THIS ERRRO [2024-06-21T16:52:55.875Z] No job functions found. Try making your job classes and methods public. If you're using binding extensions (e.g. Azure Storage, ServiceBus, Timers, etc.) make sure you've called the registration method for the extension(s) in your startup code (e.g. builder.AddAzureStorage(), builder.AddServiceBus(), builder.AddTimers(), etc.).

[ranma2913]

@phs18, I followed this example: https://learn.microsoft.com/en-us/azure/azure-functions/functions-bindings-event-hubs-trigger?tabs=python-v2%2Cisolated-process%2Cnodejs-v4%2Cfunctionsv2%2Cextensionv5&pivots=programming-language-javascript#example they don't mention anything about binder/registration.

Do you have any example code I could look at that might be more verbose or complete than the Azure Docs?

[phs18]

@ranma2913
VERBOSE.txt host.json local.settings.json requirements.txt

FUNCTION_APP.PY

import logging import azure.functions as func

app = func.FunctionApp()

@app.function_name(name="EventHubTrigger1") @app.event_hub_message_trigger(arg_name="myhub", event_hub_name="eh1", connection="connection") def test_function(myhub: func.EventHubEvent): logging.info('Python EventHub trigger processed an event: %s', myhub.get_body().decode('utf-8'))

[Goldstorm]

Command to run: func extensions install --package Microsoft.Azure.WebJobs.Extensions.EventHubs --version 6.3.0

Does not work, unfortunately. I think it can't find it in the particular extension bundle? I did use 6.3.0, and 6.3.3 fyi

First remove extensinoBundle from host.json

Then you'll be able to add the package, & it will create an extensions.csproj file like in my PR #20

This was it! So the solution to this is as follows

1. Do not use the extension bundle
2. Explicitly define the extensions by doing a manual func install. Ensure its on 6.3.X+

Ill be sure to put up a python code snippet when I get time this weekend.

[hsp0001]

@Goldstorm can you please share your settings and code for it. i am having similar issue