microsoft / promptflow

Build high-quality LLM apps - from prototyping, testing to production deployment and monitoring.
https://microsoft.github.io/promptflow/
MIT License
9.55k stars 871 forks source link

[BUG] YAML Variable Interpolation Fails for List Inputs in PromptFlow Python Nodes #3841

Open ivanko-94 opened 2 weeks ago

ivanko-94 commented 2 weeks ago

Describe the bug When attempting to pass a list to a Python node in a PromptFlow YAML file, variable references (e.g., ${inputs.num_1}) are not interpolated correctly. Whereas the same interpolation for direct function inputs is done correctly. Instead of evaluating to actual values, the List/Dict elements are parsed as strings.

How To Reproduce the bug Steps to reproduce the behavior, how frequent can you experience the bug:

  1. create a flow.dag.yaml file:
$schema: https://azuremlschemas.azureedge.net/promptflow/latest/Flow.schema.json

inputs:
  num_1:
    type: int
  num_2:
    type: int

outputs:
  out_sum1:
    type: string
    reference: ${sum1.output}
  out_sum2:
    type: string
    reference: ${sum2.output}

nodes:
  - name: sum1
    description: this node is executed correctly because list elements are static at graph parse time
    type: python
    source:
      type: code
      path: ./fn_with_list_input.py
    inputs:
      list_arg:
        - 1
        - 2

  - name: sum2
    description: this node is not executed correctly because list elements are treated as strings
    type: python
    source:
      type: code
      path: ./fn_with_list_input.py
    inputs:
      list_arg:
        - ${inputs.num_1}
        - ${inputs.num_2}
  1. create a script for the custom tool ./fn_with_list_input.py
from typing import List

from promptflow.core import tool

@tool
def sum_fn(list_arg: List):
    return sum(list_arg)
  1. create a data.jsonl
{ "num_1": 1, "num_2": 2 }
  1. main.py
from promptflow.client import PFClient

if __name__ == "__main__":

    promptflow_client = PFClient()
    promptflow_client.run(
        flow="./",
        data="./data.jsonl",
        stream=True,
    )
  1. run main.py

Expected behavior Both nodes (sum1 and sum2) should get inputs 1 and 2 and return their sum. Node sum2 should replace values for ${inputs.num_1} and ${inputs.num_2}.

Actual behavior Node sum1 executes as expected. Node sum2 throws an exception: (TypeError) unsupported operand type(s) for +: 'int' and 'str' upon further inspection in tracer PF did not replace ${inputs.num_1} with its actual value and has treated it as string:

{
"list_arg": [
0: "${inputs.num_1}",
1: "${inputs.num_2}",
]
}

Screenshots Image

Running Information(please complete the following information):