Pingora In Front Of HTTP/HTTPS Proxies

[ArielLahiany]

Trying to setup a Pingora-based logic in front of an HTTP/HTTPS proxy (Squid etc) without luck. When creating an HTTP request over cURL, the logic works as expected. When creating an HTTPS request, the following error is shown:

cURL

curl: (56) CONNECT tunnel failed, response 502

Pingora

[ERROR pingora_proxy] Fail to proxy: Upstream ConnectError context: Fail to establish CONNECT proxy: addr: 127.0.0.1:3128, scheme: HTTP,proxy: next_hop: 127.0.0.1:3128, host: 127.0.0.1, port: 3128, cause:  context: CONNECT proxy connect() error to "127.0.0.1:3128" cause:  context: Fail to connect to 127.0.0.1:3128 cause: No such file or directory (os error 2), status: 502, tries: 1, retry: false, CONNECT google.com:443, Host: google.com:443

I know it related to the fact the HTTPS proxy are working over tunnels and the CONNECT method, and I'm not trying to create an SSL MITM proxy to read the content of the request. Just to be able to use the request_filter method of the ProxyHttp trait of Pingora.

Pingora

use async_trait::async_trait;

use pingora::proxy::{http_proxy_service, ProxyHttp, Session};
use pingora::server::Server;
use pingora::upstreams::peer::HttpPeer;
use pingora::Result;

pub struct Gateway {}

#[async_trait]
impl ProxyHttp for Gateway {
    type CTX = ();
    fn new_ctx(&self) -> Self::CTX {}

    async fn upstream_peer(
        &self,
        _session: &mut Session,
        _ctx: &mut Self::CTX,
    ) -> Result<Box<HttpPeer>> {
        Ok(Box::new(HttpPeer::new(
            ("127.0.0.1", 3128),
            false,
            "".to_string(),
        )))
    }
}

fn main() {
    env_logger::init();
    let mut server = Server::new(None).unwrap();
    server.bootstrap();
    let mut service = http_proxy_service(&server.configuration, Gateway {});
    service.add_tcp("0.0.0.0:8080");
    server.add_service(service);
    server.run_forever();
}

Squid

acl SSL_ports port 443
acl Safe_ports port 80
acl Safe_ports port 443
acl CONNECT method CONNECT

http_access allow localhost manager
http_access allow localhost
http_access allow localnet
http_access deny manager
http_access deny !Safe_ports
http_access deny CONNECT !SSL_ports
http_access deny all

cache deny all

coredump_dir /var/spool/squid

logfile_rotate 10

cURL

curl -x http://localhost:8080 https://google.com
curl: (56) CONNECT tunnel failed, response 502

Would really appropriate a push in the right direction.

[andrewhavck]

Is the example you provided here complete? We shouldn't be invoking proxy_connect unless new_proxy is called or proxy is set on HttpPeer.

If I try your example code I see:

2024-04-12T22:21:26Z ERROR pingora_proxy] Fail to proxy: Upstream ConnectRefused context: Fail to connect to addr: 127.0.0.1:3128, scheme: HTTP, cause:  context: Fail to connect to 127.0.0.1:3128 cause: Connection refused (os error 61), status: 502, tries: 1, retry: false, CONNECT google.com:443, Host: google.com:443

The error log you provided is happening because right now proxy_connect expects a UDS.

[ArielLahiany]

The code snippets I've added are full. I'm using it on an MacOS Intel machine if that make any diffrence. I'm attaching here the full Pingora service logs:

[2024-04-13T14:46:16Z INFO  pingora_core::server] Bootstrap starting
[2024-04-13T14:46:16Z DEBUG pingora_core::server] None
[2024-04-13T14:46:16Z INFO  pingora_core::server] Bootstrap done
[2024-04-13T14:46:16Z INFO  pingora_core::server] Server starting
[2024-04-13T14:47:12Z DEBUG pingora_core::services::listening] new event!
[2024-04-13T14:47:12Z DEBUG pingora_proxy] Successfully get a new request
[2024-04-13T14:47:12Z DEBUG pingora_core::connectors] No reusable connection found for addr: 127.0.0.1:3128, scheme: HTTP,
[2024-04-13T14:47:12Z DEBUG pingora_core::connectors::l4] connected to new server: 127.0.0.1:3128
[2024-04-13T14:47:12Z DEBUG pingora_proxy::proxy_h1] Sending header to upstream RequestHeader { base: Parts { method: CONNECT, uri: google.com:443, version: HTTP/1.1, headers: {"host": "google.com:443", "user-agent": "curl/8.4.0", "proxy-connection": "Keep-Alive"} }, header_name_map: Some({"host": CaseHeaderName(b"Host"), "user-agent": CaseHeaderName(b"User-Agent"), "proxy-connection": CaseHeaderName(b"Proxy-Connection")}), raw_path_fallback: [] }
[2024-04-13T14:47:12Z DEBUG pingora_proxy::proxy_h1] finish sending body to upstream
[2024-04-13T14:47:12Z DEBUG pingora_core::protocols::http::v1::client] Response header: ResponseHeader { base: Parts { status: 200, version: HTTP/1.1, headers: {} }, header_name_map: Some({}) }
[2024-04-13T14:47:12Z DEBUG pingora_proxy::proxy_h1] upstream event: Some(Header(ResponseHeader { base: Parts { status: 200, version: HTTP/1.1, headers: {} }, header_name_map: Some({}) }, false))
[2024-04-13T14:47:12Z DEBUG pingora_proxy::proxy_h1] downstream event
[2024-04-13T14:47:12Z DEBUG pingora_core::protocols::l4::stream] Dropping socket BufStream { inner: BufReader { reader: BufWriter { writer: Tcp(PollEvented { io: Some(TcpStream { addr: 127.0.0.1:49747, peer: 127.0.0.1:3128, fd: 16 }) }), buffer: 0/1460, written: 0 }, buffer: 0/65536 } }
[2024-04-13T14:47:12Z WARN  pingora_core::protocols::http::v1::server] Respond header is already sent, cannot send again
[2024-04-13T14:47:12Z ERROR pingora_proxy] Fail to proxy: Downstream ConnectError context: Peer: addr: 127.0.0.1:3128, scheme: HTTP, cause:  context: Sent data after end of body, status: 400, tries: 1, retry: false, CONNECT google.com:443, Host: google.com:443
[2024-04-13T14:47:12Z DEBUG pingora_core::protocols::l4::stream] Dropping socket BufStream { inner: BufReader { reader: BufWriter { writer: Tcp(PollEvented { io: Some(TcpStream { addr: 127.0.0.1:8080, peer: 127.0.0.1:49746, fd: 15 }) }), buffer: 0/1460, written: 0 }, buffer: 319/65536 } }

And the cURL command resulted in:

curl: (35) LibreSSL SSL_connect: SSL_ERROR_SYSCALL in connection to google.com:443

Should I configure Pingora differently? I was manage to make it with in duplex mode with the example from here but then I can't use the request_filter method.

[vicanso]

I try to proxy http://localhost:6018/ to https://www.baidu.com/, set HttpPeer::new(("120.232.145.185", 443), true, "www.baidu.com").

[andrewhavck]

Circling back to this it looks like there's an issue handling CONNECT responses.

[2024-04-13T14:47:12Z WARN  pingora_core::protocols::http::v1::server] Respond header is already sent, cannot send again

[Abdullah13521]

I’m facing the same issue, were you able to figure it out?

[ArielLahiany]

I’m facing the same issue, were you able to figure it out?

Unfortunately no. The options as I see them are:

1. This logic is not supported by Pingora.
2. The docs are not showing a valid example for this scenario. Either way, no success so far. Left our HAProxy + Lua at the moment.

[futurist]

Another use case is to setup a local dev proxy to run a http dev server like http://127.0.0.1:3000 (with response body Hello World!), then setup pingora to proxy a online https domain like https://example.com, when visit https://example.com the upstream will redirect to http://127.0.0.1:3000. The curl command will be:

curl -x http://127.0.0.1:8080 https://example.com

And the response will be Hello World!