Azure / azure-functions-python-worker

Python worker for Azure Functions.
http://aka.ms/azurefunctions
MIT License
331 stars 100 forks source link

[Bug] Can't flush stdout at shutdown #1505

Open Halpph opened 3 weeks ago

Halpph commented 3 weeks ago

Expected Behavior

Hello everyone, I have some azure functions that were working for long time and now they stopped, also locally they work fine, so I tried to investigate for the error, I think the error is due to the fact that I created an AzureLogHandler, that's supposed to log some informations to my Azure Log Analytics and the flush is not well supported. I see that your code is calling sys.stdout.flush() here but I guess that's not the right way, maybe it should be done calling logging.shutdown(), so that it doesn't fail.

Actual Behavior

The Azure function is running fine, but at the end I get this error: Result: Failure Exception: AttributeError: 'PrintLogger' object has no attribute 'flush' Stack: File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 643, in _handle__invocation_request sys.stdout.flush() I'd like it to stop throwing this error.

Steps to Reproduce

No response

Relevant code being tried

from collections import namedtuple
import json
import os
import datetime
from laura.powerbi_pilot import PowerBIPilot
import logging
import azure.functions as func
from opencensus.ext.azure.log_exporter import AzureLogHandler

def get_powerbi_vars(workspace_id):
    powerbi_vars = namedtuple("powerbi_vars", [
        'appid',
        'password',
        'tenant_id',
        'workspace_id'
    ])

    t = [os.environ['PowerBiSPAppId'],
         os.environ['PowerBiSPSecretId'],
         os.environ['TenantId'],
         workspace_id
         ]

    powerbi_vars = powerbi_vars._make(t)
    return powerbi_vars

def main(logdatasetrefreshtimer: func.TimerRequest) -> None:
    utc_timestamp = datetime.datetime.utcnow().replace(
        tzinfo=datetime.timezone.utc).isoformat()

    if logdatasetrefreshtimer.past_due:
        logging.info('The timer is past due!')

    data_logger = logging.getLogger("data_logging")
    data_logger.setLevel(logging.INFO)

    logging.info('Fetching APPINSIGHTS key')
    app_insights_instr_key = os.environ["APPINSIGHTS_INSTRUMENTATIONKEY"]

    logging.info('Adding key to handler')
    data_logger.addHandler(AzureLogHandler(
        connection_string=f"InstrumentationKey={app_insights_instr_key}")
    )

    workspaces = list(os.environ["PowerBiWorkspaces"].split(","))

    for workspace in workspaces:
        logging.info(f"~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ Workspace: {workspace}~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~")
        powerbi_pilot = PowerBIPilot(
            powerbi_vars=get_powerbi_vars(workspace_id=workspace))
        try:
            powerbi_pilot.get_token()
            datasets = (json.loads(powerbi_pilot.list_datasets_from_group(powerbi_pilot.workspace_id).content)['value'])
            #logging.info(datasets)
            for dataset in datasets:
                if dataset['name'] not in ["Report Usage Metrics Model","Usage Metrics Report"]:
                    logging.info(
                        f"~~~~~~~~~~~~~~ Refresh history dataset:  {dataset['id']}: {dataset['name']} ~~~~~~~~~~~~~~")
                    for refresh in (json.loads(powerbi_pilot.get_refresh_history(dataset['id']).content)['value']):
                        refresh['dataset'] = dataset['id']
                        refresh['workspace'] = workspace
                        data_logger.info(json.dumps(refresh))

        except Exception as e:
            logging.info(e)

Relevant log output

06/07/2024, 06:00:09.452
Error
Result: Failure Exception: AttributeError: 'PrintLogger' object has no attribute 'flush' Stack: File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 643, in _handle__invocation_request sys.stdout.flush()
06/07/2024, 06:00:09.452
Error
Executed 'Functions.PowerBiLogger' (Failed, Id=6799712d-8639-48e6-85c4-5a506d2a99ab, Duration=9449ms)
06/07/2024, 06:00:09.460
Error
Result: Failure Exception: AttributeError: 'PrintLogger' object has no attribute 'flush' Stack: File "/azure-functions-host/workers/python/3.9/LINUX/X64/azure_functions_worker/dispatcher.py", line 643, in _handle__invocation_request sys.stdout.flush()

requirements.txt file

azure-functions
requests
databricks-cli
opencensus-ext-azure
pandas
./domains/data_tamers/functions/shared/wheels/laura-0.22.0-py3-none-any.whl

Where are you facing this problem?

Production Environment (explain below)

Function app name

No response

Additional Information

No response

bhagyshricompany commented 6 days ago

@gavin-aguiar please comment and validate.Thanks