Azure / azure-functions-python-worker

Python worker for Azure Functions.
http://aka.ms/azurefunctions
MIT License
331 stars 100 forks source link

Event Hub Trigger using app config for connection #1346

Closed thedassi closed 6 months ago

thedassi commented 8 months ago

Is your question related to a specific version? If so, please specify:

Azure Function V2 ExtensionBundle 3.*, 4.0.0

What binding does your question apply to, if any? (e.g. Blob Trigger, Event Hub Binding, etc)

Event Hub Message Trigger Event Hub Output

Question

Is there a way to let event hub message trigger read the connection from app config, im running my function locally and want to read the event hub connection from app config instead of local.settings.json and also when i deploy it into production. Currently i have some custom code that uses the AzureAppConfigurationClient to read configuration out of app config and set them to os.environ variables.

Take note that I'm not the best when it comes to python development so i might be missing something simple. But i keep on getting an error when i try and run my function saying:

Microsoft.Azure.WebJobs.Extensions.EventHubs: EventHub account connection string 'EventHub' does not exist.Make sure that it is a defined App Setting.

I have tried to change connection="EventHub" to connection=os.environ["EventHub"] but then it tries and fiends my evnet hub url as an setting name.

Here i have my azureAppConfiguration.py that reads the settings

import os
import logging
from azure.appconfiguration import AzureAppConfigurationClient
from azure.identity import DefaultAzureCredential

def add_azure_app_configuration(prefix: str) -> None:
    if "ConnectionStrings:AppConfiguration" in os.environ:
        connection_str = os.environ["ConnectionStrings:AppConfiguration"]
        appconfig = AzureAppConfigurationClient.from_connection_string(
            connection_str)

        _add_config_to_environ(appconfig, prefix)
        logging.warning(os.environ)

    if "AppConfigurationEndpoint" in os.environ:

        credential = DefaultAzureCredential()
        connection_str = os.environ["AppConfigurationEndpoint"]

        appconfig = AzureAppConfigurationClient(
            base_url=connection_str, credential=credential)

        _add_config_to_environ(appconfig, prefix)
        logging.warning(os.environ)

def _add_config_to_environ(appConfig: AzureAppConfigurationClient, prefix: str):
    allConfig = appConfig.list_configuration_settings(
        key_filter=f"{prefix}:*")
    for config in allConfig:
        os.environ[config.key[len(f"{prefix}:"):]] = config.value

here is my function_app.py code

import os
from Functions.SolverEventHubFunction import app_bp as solverFunction
from Helpers.azureAppConfiguration import add_azure_app_configuration
import azure.functions as func

app = func.FunctionApp()

add_azure_app_configuration("Solver")

app.register_functions(solverFunction)

**here is my actual function code**

import os
from Infrastructure.solver_service import calculate
import json
import azure.functions as func

app_bp = func.Blueprint()

@app_bp.function_name(name="SolverEventHubFunction")
@app_bp.event_hub_message_trigger(arg_name="eventInput",
                                  event_hub_name=os.environ["EventHubNameRead"],
                                  connection="EventHub")
@app_bp.event_hub_output(arg_name="eventOutput",
                         event_hub_name=os.environ["EventHubNameWrite"],
                         connection="EventHub")
def main(eventInput: func.EventHubEvent, eventOutput: func.Out[str]) -> None:
    event_data = json.loads(eventInput.get_body().decode('utf-8'))
    customerId = event_data['CustomerId']
    results = calculate(customerId)
    eventOutput.set(results.to_json(orient='records'))
bhagyshricompany commented 8 months ago

Thanks for reporting. will check and update.pls share the all steps with req.txt file. pls follow this doc. https://learn.microsoft.com/en-us/azure/event-hubs/ https://learn.microsoft.com/en-us/azure/azure-functions/functions-reference-python?tabs=asgi%2Capplication-level&pivots=python-mode-decorators https://learn.microsoft.com/en-us/azure/azure-app-configuration/

thedassi commented 8 months ago

I used all those links to get to where i am now, i have attached all my files to this comment requirements.txt function_app.py.txt SolverEventHubFunction.py.txt azureAppConfiguration.py.txt

bhagyshricompany commented 7 months ago

@gavin-aguiar pls comment and validate.Thanks

hallvictoria commented 6 months ago

Hi @thedassi , thanks for your patience with this.

For getting configuration settings with Azure App Configuration, please refer to this guide: https://learn.microsoft.com/en-us/azure/azure-app-configuration/quickstart-python?tabs=windowscommandprompt

You can get the AzureAppConfigurationClient using from_connection_string and use get_configuration_setting to obtain the EventHub connection string.

When deploying the app, the connection string does need to be set as an app setting.

Please let me know if you have any further issues.