HTTP Request Node Accessing HTTPS with Non-443 Port Returns 403 Error

wayneg123 commented 1 month ago

Self Checks

[X] This is only for bug report, if you would like to ask a question, please head to Discussions.
[X] I have searched for existing issues search for existing issues, including closed ones.
[X] I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
[X] 请务必使用英文提交 Issue，否则会被关闭。谢谢！:）
[X] Please do not modify this template :) and fill in all the required fields.

Dify version

0.6.14

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

Use the HTTP Request node.
Set the API method to GET.
Set the URL to https://www2.medicareaustralia.gov.au:5447/.
Send the request.

Using requests in code node will cause 403 error too.

✔️ Expected Behavior

The request should successfully connect to the server and retrieve the expected response

❌ Actual Behavior

The request returns a 403 Forbidden error code.

Error message from code node.

ov.au', port=5447): Max retries exceeded with url: / (Caused by ProxyError('Unable to connect to proxy', OSError('Tunnel connection failed: 403 Forbidden'))) 'Unable to connect to proxy', OSError('Tunnel connection failed: 403 Forbidden'))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/tmp/code/65ce1c33_503d_4fa8_897c_0d29d57c8ea2.py", line 65, in <module> File "/tmp/code/65ce1c33_503d_4fa8_897c_0d29d57c8ea2.py", line 54, in main File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 73, in get File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 694, in send requests.exceptions.ProxyError: HTTPSConnectionPool(host='www2.medicareaustralia.gov.au', port=5447): Max retries exceeded with url: / (Caused by ProxyError('Unable to connect to proxy', OSError('Tunnel connection failed: 403 Forbidden'))) 'Unable to connect to proxy', OSError('Tunnel connection failed: 403 Forbidden'))) During handling of the above exception, another exception occurred: Traceback (most recent call last): File "/tmp/code/65ce1c33_503d_4fa8_897c_0d29d57c8ea2.py", line 65, in <module> File "/tmp/code/65ce1c33_503d_4fa8_897c_0d29d57c8ea2.py", line 54, in main File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 73, in get File "/usr/local/lib/python3.10/site-packages/requests/api.py", line 59, in request File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 589, in request File "/usr/local/lib/python3.10/site-packages/requests/sessions.py", line 703, in send File "/usr/local/lib/python3.10/site-packages/requests/adapters.py", line 694, in send requests.exceptions.ProxyError: HTTPSConnectionPool(host='www2.medicareaustralia.gov.au', port=5447): Max retries exceeded with url: / (Caused by ProxyError('Unable to connect to proxy', OSError('Tunnel connection failed: 403 Forbidden'))) error: exit status 255

dosubot[bot] commented 1 month ago

Hello @wayneg123! I'm here to help you with any bugs, questions, or contributions you have for the repository. Let's work together to solve this issue!

To resolve the 403 Forbidden error when making an HTTPS request to a non-standard port (5447) using the HTTP Request node or a Python script with the requests library, you need to ensure that the request is properly authorized and that the server is configured to accept requests on that port.

Using HTTP Request Node

Ensure that the authorization configuration is correctly set. The default configuration in the provided code uses "type": "no-auth", which means no authorization is being used. If the server requires authorization, you need to update this configuration.

Example:

@classmethod
def get_default_config(cls) -> dict:
    return {
        "type": "http-request",
        "config": {
            "method": "get",
            "authorization": {
                "type": "basic",  # or "bearer", "api-key", etc.
                "username": "your_username",
                "password": "your_password"
            },
            "body": {
                "type": "none"
            },
            "timeout": {
                **HTTP_REQUEST_DEFAULT_TIMEOUT.model_dump(),
                "max_connect_timeout": MAX_CONNECT_TIMEOUT,
                "max_read_timeout": MAX_READ_TIMEOUT,
                "max_write_timeout": MAX_WRITE_TIMEOUT,
            }
        },
    }

Using Python `requests` Library

Ensure that you are including the necessary headers and authorization in your request. Here is an example of how to do this:

import requests

url = "https://yourserver.com:5447/your-endpoint"
headers = {
    "Authorization": "Bearer your_token",  # or other authorization method
    "Content-Type": "application/json"
}

response = requests.get(url, headers=headers)

if response.status_code == 403:
    print("Access forbidden: Check your authorization and server configuration.")
else:
    print("Response:", response.content)

Make sure to replace "Bearer your_token" with the appropriate authorization method required by your server ^[1].

To continue talking to Dosu, mention @dosu.

crazywoola commented 1 month ago

There are few options that you can choose. The simplest way is you can turn of the ssrf_proxy by setting


# SSRF Proxy server HTTP URL
SSRF_PROXY_HTTP_URL=http://ssrf_proxy:3128 => SSRF_PROXY_HTTP_URL=
# SSRF Proxy server HTTPS URL
SSRF_PROXY_HTTPS_URL=http://ssrf_proxy:3128 => SSRF_PROXY_HTTPS_URL=

Which will turn off the proxy.

iiiusky commented 1 month ago

Has the issue been resolved? I encountered the same problem, showing a 403 error.I think it's possible to add a parameter to ignore SSL in the HTTP request section.

wayneg123 commented 1 month ago

I've found the solution. Simply add the following line to docker/ssrf_proxy/squid.conf.template, and it works:

acl SSL_ports port YOUR_HTTPS_PORT

Modas-Li commented 1 month ago

I've found the solution. Simply add the following line to docker/ssrf_proxy/squid.conf.template, and it works:
acl SSL_ports port YOUR_HTTPS_PORT

then found issue:[SSL: UNSAFE_LEGACY_RENEGOTIATION_DISABLED] unsafe legacy renegotiation disabled (ssl.c:1007) help!

langgenius / dify