The Iteration node stucks when the output is Array[Array[File]]

[StdioA]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.15

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

1. Create an Iteration block
2. Add a block which will produce Array[File] output, such as Stable Diffusion
3. Set the files output of Stable Diffusion to the output variable of iteraion block, thus it will produce an output with Array[Array[File]] type theoretically
4. Use the iteration output in another block, as you may notify that the type of iteration output becomes Array[string]
5. Run the workflow

✔️ Expected Behavior

The workflow runs normally, as I may flatten the nested array with Python or Jinja renderer after iteration.

Or maybe there's a way to flatten the Array[File] inside the loop body.

❌ Actual Behavior

The workflow stucks at the iteration procedure.

[dosubot[bot]]

@StdioA Hello there! I'm here to help you with any bugs, questions, or contributions you have for the repository. Just give me a shout, and I'll do my best to assist you while we wait for a human maintainer.

The Iteration node in Dify version 0.6.15 has known issues when dealing with nested arrays, such as Array[Array[File]]. Specifically, the Iteration node expects the iterator to be a simple list and raises a ValueError if it is not:

if not isinstance(iterator, list):
    raise ValueError(f"Invalid iterator value: {iterator}, please provide a list.")

This means that nested arrays or other complex types like Array[File] will not be handled correctly by the current implementation ^[1].

Additionally, the Iteration node treats VarType.arrayFile as VarType.object, which might cause unexpected behavior when processing nested arrays ^[2].

To debug why the Iteration node gets stuck, you can utilize the available logs and debugging tools. The iteration_node.py file provides detailed methods for iteration logic and state management, which can be useful for adding logging and tracing the execution flow ^[1].

For a workaround, you might need to preprocess the nested arrays to flatten them or convert them into a format that the Iteration node can handle correctly before passing them into the Iteration node.

To continue talking to Dosu, mention @dosu.

[StdioA]

@dosu

I have tried to add an Python code execution node to flatten the SD output, or even translate the File object into a URL string as a workaround before I open this issue.
However, I couldn't add the files output from SD into the code argument list. Maybe the File type is not fully supported in the workflow yet.

[dosubot[bot]]

To handle the File type correctly in the workflow or flatten nested arrays before passing them into the Iteration node in Dify version 0.6.15, you can follow these steps:

1. Ensure Input is in Array Format: Before using the Iteration node, make sure that the input value is formatted as an array. This can be done using a parameter extraction node or a code node to convert the input into an array structure.

2. Configure Iteration Node: The Iteration node will execute the same steps for each item in the array. You need to configure the Iteration node to handle the array items correctly.

3. Handle Nested Arrays: If you have nested arrays, you can use a code node to flatten them before passing them into the Iteration node. Here is an example of how to flatten nested arrays using a code node:

   def flatten(nested_list):
       flat_list = []
       for item in nested_list:
           if isinstance(item, list):
               flat_list.extend(flatten(item))
           else:
               flat_list.append(item)
       return flat_list
   
   def main(input_array):
       return {
           "flattened_array": flatten(input_array)
       }

4. Process Files: If you are dealing with File types, ensure that the files are correctly handled and passed into the Iteration node. You can use the useConfig function to manage the configuration and execution of iteration nodes, including handling arrays and File types.

Here is an example of how to configure the Iteration node and handle the input data:

const handleRun = useCallback((data: Record<string, any>) => {
  const formattedData: Record<string, any> = {}
  Object.keys(allVarObject).forEach((key) => {
    const [varSectorStr, nodeId] = key.split(DELIMITER)
    formattedData[`${nodeId}.${allVarObject[key].inSingleRunPassedKey}`] = data[varSectorStr]
  })
  formattedData[iteratorInputKey] = data[iteratorInputKey]
  doHandleRun(formattedData)
}, [allVarObject, doHandleRun, iteratorInputKey])

const inputVarValues = (() => {
  const vars: Record<string, any> = {}
  Object.keys(runInputData)
    .filter(key => ![iteratorInputKey].includes(key))
    .forEach((key) => {
      vars[key] = runInputData[key]
    })
  return vars
})()

const setInputVarValues = useCallback((newPayload: Record<string, any>) => {
  const newVars = {
    ...newPayload,
    [iteratorInputKey]: runInputData[iteratorInputKey],
  }
  setRunInputData(newVars)
}, [iteratorInputKey, runInputData, setRunInputData])

const iterator = runInputData[iteratorInputKey]
const setIterator = useCallback((newIterator: string[]) => {
  setRunInputData({
    ...runInputData,
    [iteratorInputKey]: newIterator,
  })
}, [iteratorInputKey, runInputData, setRunInputData])

This function ensures that the File type and other nested arrays are correctly handled and passed into the Iteration node. The handleRun function formats the data appropriately, and the setIterator function manages the iteration input key, ensuring that nested arrays are flattened and processed correctly ^[1][2][3].

For more detailed examples and configurations, you can refer to the Dify documentation on iteration nodes and handling arrays.

To continue talking to Dosu, mention @dosu.