[Feature Request] Allow configuration of log format

[FriederikeBrezing]

Is your feature request related to a problem? Please describe. We are exporting logs to datadog, and are planning to inject OTEL trace information into our logs. This requires that logs use a certain format. We can configure the format of logs in our custom code but the log format for logs from the promptflow executor or serving app use their own format. Examples: 2024-04-17 09:52:48 +0100 8826 execution.flow INFO Start executing nodes in thread pool mod [2024-04-17 09:51:16,380][pfserving-app][INFO] - Promptflow executor starts initializing...

Describe the solution you'd like An option and clear instructions for how to customise the log format of logs created by promptflow. Ideally a way to provide a log config file which will be used by promptflow to configure logging so what we can get it to use any log format or handlers.

Describe alternatives you've considered We tried to wrap calls to promptflow in a python app and configured the logging before calling promptflow but this did not affect the logging configuration used by promptflow.

[PeiwenGaoMS]

Hi @FriederikeBrezing ,

Thank you for your feedback. Based on the issues you raised, we have thought of three potential solutions:

• Add environment variables in the flow yaml, for example:

  
  inputs:
  text:
  type: string
  outputs:
  output_text:
  type: string
  reference: ${print_input.output}
  nodes:

• name: print_input type: python source: type: code path: print_input.py inputs: text: ${inputs.text} environment_variables: PF_LOG_FORMAT: "[%(asctime)s][%(name)s][%(levelname)s] - %(message)s" PF_LOG_DATETIME_FORMAT: "%Y/%m/%d %H:%M:%S"

- Pass the format you need through a function provided before submitting the flow, for example:
``` python
from promptflow.core import set_log_format
from promptflow.client import PFClient

set_log_format(fmt=..., datefmt=...)

pf = PFClient()
flow_path = ...
flow_inputs = ...
flow_result = pf.test(flow=flow_path, inputs=flow_inputs)

• Place a log_config.json in the flow folder, for example:

  {
  "log_format": "[%(asctime)s][%(name)s][%(levelname)s] - %(message)s",
  "date_format": "%Y/%m/%d %H:%M:%S"
  }

Based on the three options provided, we will change the format of the promptflow logger after receiving the format you pass. Which of these three solutions would you prefer?

[FriederikeBrezing]

Thanks for this! Checked with other team members, and we agree that we'd prefer the 3rd option (log_config.json), it seems to be the most flexible one and keeps logging configuration separate from the flow definition.

[PeiwenGaoMS]

Hi @FriederikeBrezing ,

Sorry for not carefully considering the feasibility of the solutions yesterday. After further evaluation, we have concluded that the third option is not feasible. Among the first two options, we recommend the first one because it seamlessly integrates with both the Promptflow VSCode Extension and the Azure Machine Learning Workspace for submitting workflows.

Regarding your suggestion to separate the flow definition and define the log format, setting these environment variables directly in the system environment also works well, eliminating the need to include them in the flow yaml.

[FriederikeBrezing]

Yes, if option 3 is not possible then we'd prefer option 1.

[PeiwenGaoMS]

I'll add these environment variables to enable log format configuration and will update you once it's published. Thanks!

[camilla11]

Hi, we are also looking to change the formatting of the logs. We also need to able to ensure the logs use JSON log formatter from json_log_formatter.JSONFormatterwhich we set on all loggers when starting the app but look to be overwritten here: https://github.com/microsoft/promptflow/blob/518af95d986df0201e84881ad10572686fbd404f/src/promptflow-core/promptflow/_utils/logger_utils.py#L183 I'm not sure if there is a way I'm missing to keep the formatting we set on app initialization.

[PeiwenGaoMS]

Hi @camilla11 @FriederikeBrezing , I've merged the PR into the main branch, and the changes will be included in the next version of promptflow (1.11.0). After the new version is released, you can refer to the doc to use environment variables to configure the formatter you need.

[camilla11]

@PeiwenGaoMS do you know if its possible to add Json formatting to the logs, especially the exceptions?

[PeiwenGaoMS]

@camilla11 I think this is a new feature request. Please open a new issue so we can track it accordingly.