When running Python code through a code execution tool, passing in an excessively long string can cause parsing to fail.

[wdmzjxzl]

Self Checks

• [X] This is only for bug report, if you would like to ask a question, please head to Discussions.
• [X] I have searched for existing issues search for existing issues, including closed ones.
• [X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• [X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.11

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

After accessing an interface using an HTTP request to get data, I passed the body as a variable to a code execution tool (Python) and used Python's json library to parse the JSON data. However, the variable string is too long, approximately 1 million in length. I adjusted the CODE_MAX_STRING_LENGTH value in the docker-compose.yaml file, but it still doesn't take effect. Why is this happening? How should I handle this situation? error_msg is：Node 代码执行 run failed: Failed to parse result

✔️ Expected Behavior

I hope to pass in the extremely long string and still be able to parse it normally in Python code.

❌ Actual Behavior

An error occurred, and the Python code could not execute properly.

[dosubot[bot]]

Hello, @wdmzjxzl! I'm here to help you with any bugs, questions, or contributions you may have for the repository. Let's work together to solve this issue!

To handle an excessively long string (approximately 1 million characters) in Python code using Dify version 0.6.11, self-hosted with Docker, follow these steps:

1. Adjust the CODE_MAX_STRING_LENGTH Environment Variable: Ensure that the CODE_MAX_STRING_LENGTH environment variable is set appropriately in your Docker configuration to handle the long string. By default, it is set to 10,000 characters, so you need to increase this limit. Update your docker-compose.yaml file as follows:

   environment:
    - CODE_MAX_STRING_LENGTH=1000000

2. Verify Docker Configuration: After updating the docker-compose.yaml file, make sure to restart your Docker containers to apply the changes:

   docker-compose down
   docker-compose up -d

3. Efficient String Processing in Python: Use the Code node to handle the string processing efficiently. Here is an example of how you can achieve this:

   def main(long_string: str) -> str:
      # Perform your string processing here
      # For example, let's just return the length of the string
      return str(len(long_string))

   This code snippet defines a main function that takes a long string as input and returns its length. You can replace the string processing logic with your specific requirements.

By following these steps, you should be able to handle and parse the excessively long string without encountering the "Failed to parse result" error ^[1][2].

To continue talking to Dosu, mention @dosu.

[wdmzjxzl]

@dosu The Python code in the code execution tool is:

def main(arg1): import json return { "result": json.loads(arg1) } The input variable is the string from the HTTP response body, which is a string in the format list[dict]. I need to parse it using Python.

However, when I execute json.loads(arg1), I get the error: Node code execution run failed: Failed to parse result.

But when I execute str(len(long_string)), I am able to print out the length of the string, so it might not be an issue of the input string being too long. How should I solve this problem?

[dosubot[bot]]

The issue with json.loads(arg1) might be due to the sandboxed execution environment in Dify, which could restrict certain functionalities. Here are steps to troubleshoot and potentially resolve the issue:

1. Check the Structure of arg1: Ensure that arg1 is a valid JSON string. Add a print statement to log the content of arg1 before parsing it.

2. Error Handling: Add error handling to catch and log any exceptions during JSON parsing. This can help identify if there is a specific issue with the JSON content.

Here is an updated version of your code with additional logging and error handling:

def main(arg1):
    import json
    try:
        parsed_result = json.loads(arg1)
        return {
            "result": parsed_result
        }
    except json.JSONDecodeError as e:
        return {
            "error": str(e),
            "input_length": len(arg1),
            "input_sample": arg1[:1000]  # Log the first 1000 characters of the input for debugging
        }

This code will return an error message if json.loads(arg1) fails, along with the length of the input string and a sample of the input content. This information can help diagnose whether the issue is with the JSON content or the execution environment.

Additionally, ensure that your environment is correctly set up to use the CodeExecutor and Python3CodeProvider components. The Dify codebase includes specific constraints on string lengths and object depths, which might be relevant. For example, the MAX_STRING_LENGTH is set to 80000 characters, and the maximum depth for nested objects is 5. If your JSON string exceeds these limits, it could cause the parsing to fail. You can check these constraints in the code_node.py file:

MAX_STRING_LENGTH = int(os.environ.get('CODE_MAX_STRING_LENGTH', '80000'))
MAX_DEPTH = 5

Ensure that your JSON string and its structure comply with these constraints. If necessary, adjust these environment variables in your Docker setup to accommodate larger inputs ^[1][2][3].

To continue talking to Dosu, mention @dosu.

[wdmzjxzl]

@dosu I don't want the code to run in a sandbox environment, what should I do?

[wdmzjxzl]

@dosu In the code execution tool, my code body looks like this:

def main(arg1): return { "result": arg1 } I set the input variable as a string with a length of over 700,000 characters. Then, I use this method to return the passed variable. The output variable is result. After executing the code, the console still reports an error: Node code execution run failed: Failed to parse result.

[shengbox]

I encountered the same issue with the Book Translation example on https://cloud.dify.ai/. It occurs occasionally, with an error appearing roughly once every 10 runs.

[dosubot[bot]]

To disable the sandbox environment for code execution in Dify and handle the long string, follow these steps:

1. Disable the Sandbox: Ensure that the sandbox environment is disabled. This can be done by not starting the sandbox service with Docker. If you have previously set up a sandbox service, stop it using:

   docker-compose -f docker-compose.middleware.yaml down

2. Adjust Environment Variables: Set the CODE_MAX_STRING_LENGTH environment variable to a value greater than 700,000 characters. You can do this in your docker-compose.yaml file:

   environment:
    - CODE_MAX_STRING_LENGTH=1000000

3. Restart Docker Containers: After making changes to the docker-compose.yaml file, restart your Docker containers to apply the changes:

   docker-compose down
   docker-compose up -d

4. Update Your Code: Ensure your code handles the long string correctly. Here is an example of your code:

   def main(arg1):
      return {
          "result": arg1
      }

By following these steps, you should be able to handle the long string without encountering the "Failed to parse result" error. Disabling the sandbox environment will allow your code to run without the restrictions imposed by the sandbox, but be aware of the potential security implications and take necessary precautions ^[1][2].

To continue talking to Dosu, mention @dosu.

[shengbox]

I did not use the Docker environment; instead, I encountered the same issue while using https://cloud.dify.ai. Additionally, the same data does not always result in an error. The probability of errors increases as the string length approaches or exceeds 400,000.

[crazywoola]

We do not suggest passing a such long context to our code node, because it's meant to be a lightweight solution. You should breakdown the complex tasks into smaller one.

[achenglike]

We do not suggest passing a such long context to our code node, because it's meant to be a lightweight solution. You should breakdown the complex tasks into smaller one.

It's not that the code is long, but that the returned data is long.