Error Processing Requests Containing curl Commands in Code Blocks

daigo38 commented 1 month ago

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] [FOR CHINESE USERS] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.8.3

Cloud or Self Hosted

Cloud

Steps to reproduce

Open the Web UI or use the API.
Submit a request that contains a curl command within a code block, such as:
```
curl -X hello world
```
Observe that the request is not processed.

API Response:

Response {
  status: 403,
  statusText: 'Forbidden',
  headers: Headers {
    date: 'Thu, 26 Sep 2024 05:57:18 GMT',
    'content-type': 'text/html; charset=UTF-8',
    'transfer-encoding': 'chunked',
    connection: 'keep-alive',
    'x-frame-options': 'SAMEORIGIN, DENY',
    'referrer-policy': 'same-origin',
    'cache-control': 'max-age=15',
    expires: 'Thu, 26 Sep 2024 05:57:33 GMT',
    'report-to': '{"endpoints":[{"url":"https:\\/\\/a.nel.cloudflare.com\\/report\\/v4?s=23xvFL%2F%2FN2D77cVRqkH9%2FyNEYZMh1hThyhJHI7WQZNcXZ0FIlKMSoO%2FirUvxB%2B4zwPxjQ2J3CHxzEULSbnLNK3F9EwWbDMh2ZZPuF2EfB9S5wWCdaYhdiHMX5VjV"}],"group":"cf-nel","max_age":604800}',
    nel: '{"success_fraction":0,"report_to":"cf-nel","max_age":604800}',
    vary: 'Accept-Encoding',
    'strict-transport-security': 'max-age=31536000; includeSubDomains',
    'x-content-type-options': 'nosniff',
    server: 'cloudflare',
    'cf-ray': '8c9104046a211a19-KIX',
    'content-encoding': 'br'
  },
  body: ReadableStream { locked: false, state: 'readable', supportsBYOB: true },
  bodyUsed: false,
  ok: false,
  redirected: false,
  type: 'basic',
  url: 'https://api.dify.ai/v1/chat-messages'
}

✔️ Expected Behavior

Requests containing curl commands within code blocks should be processed successfully, or a clear error message should be displayed in the Web UI explaining why the request cannot be processed.

❌ Actual Behavior

Requests that include curl commands in a code block fail to be processed, with no error message displayed in the Web UI. When using the API, the request returns a 403 Forbidden error.

dosubot[bot] commented 1 month ago

@daigo38 Hello there! I'm here to help you with any bugs, questions, or contributions. Let me know how I can assist you!

Requests containing curl commands within code blocks fail to be processed in Dify version 0.8.3, returning a 403 Forbidden error without displaying an error message in the Web UI because the execute_code method in the CodeExecutor class does not handle 403 status codes explicitly. The method only checks for 503 and other non-200 status codes, raising a generic exception for them. This means that a 403 error does not provide a specific error message, leading to the lack of an error message in the Web UI.

Here is the relevant part of the code:

try:
    response = post(
        str(url),
        json=data,
        headers=headers,
        timeout=Timeout(
            connect=dify_config.CODE_EXECUTION_CONNECT_TIMEOUT,
            read=dify_config.CODE_EXECUTION_READ_TIMEOUT,
            write=dify_config.CODE_EXECUTION_WRITE_TIMEOUT,
            pool=None,
        ),
    )
    if response.status_code == 503:
        raise CodeExecutionError("Code execution service is unavailable")
    elif response.status_code != 200:
        raise Exception(
            f"Failed to execute code, got status code {response.status_code},"
            f" please check if the sandbox service is running"
        )
except CodeExecutionError as e:
    raise e
except Exception as e:
    raise CodeExecutionError(
        "Failed to execute code, which is likely a network issue,"
        " please check if the sandbox service is running."
        f" ( Error: {str(e)} )"
    )

To handle 403 errors explicitly, you could modify the code to include a specific check for the 403 status code and raise a more descriptive error:

try:
    response = post(
        str(url),
        json=data,
        headers=headers,
        timeout=Timeout(
            connect=dify_config.CODE_EXECUTION_CONNECT_TIMEOUT,
            read=dify_config.CODE_EXECUTION_READ_TIMEOUT,
            write=dify_config.CODE_EXECUTION_WRITE_TIMEOUT,
            pool=None,
        ),
    )
    if response.status_code == 403:
        raise CodeExecutionError("Forbidden: You don't have permission to access this resource.")
    elif response.status_code == 503:
        raise CodeExecutionError("Code execution service is unavailable")
    elif response.status_code != 200:
        raise Exception(
            f"Failed to execute code, got status code {response.status_code},"
            f" please check if the sandbox service is running"
        )
except CodeExecutionError as e:
    raise e
except Exception as e:
    raise CodeExecutionError(
        "Failed to execute code, which is likely a network issue,"
        " please check if the sandbox service is running."
        f" ( Error: {str(e)} )"
    )

This change will ensure that a more descriptive error message is provided when a 403 Forbidden error occurs ^[1].

To continue talking to Dosu, mention @dosu.

crazywoola commented 1 month ago

I think it's block by the Cloudflare side, it will ban some dangerous actions that contain harmful content like sql injections and direct curl command.

This command is filtered by that and it seems haven't got into our system yet. So we can do nothing about this.

langgenius / dify