Examples not reproducible

[JosiahParry]

Describe the bug

The examples are not very useful for individuals trying to get started. The first challenge is getting the examples to run locally in their own crate.

None of the examples use use pingora::prelude::*; as suggested in the Quick Start guide.

After swapping each individual crate to pingora:: e.g. pingora_http::RequestHeader to pingora::http::RequestHeader each of the examples always result in the following error

[2024-11-09T19:47:04Z ERROR pingora_proxy] Fail to proxy: Upstream ConnectionClosed context: Peer: addr: 1.1.1.1:443, scheme: HTTPS,sni: one.one.one.one, cause:  context: while reading response headers, bytes already read: 0, status: 502, tries: 1, retry: false, GET /, Host: localhost:6190

I suspect this is because there is not a TSL cert.

Using cargo example can show you the output of the example but it cannot get you started with a standalone example.

Pingora info

Please include the following information about your environment:

Pingora version: "0.4.0", source = "registry+https://github.com/rust-lang/crates.io-index", checksum = "79c9fc7098dc3e7d09d2d1647921005be9301cf68536826195dc5369e05124bd" Rust version: rustc 1.83.0-nightly (adf8d168a 2024-09-08) Operating system version: e.g. Apple M1 macOS 15.0.1

Steps to reproduce

Please provide step-by-step instructions to reproduce the issue. Include any relevant code snippets.

cargo new test-pingora
cd test-pingora
cargo add pingora

Copy and past the gateway example from https://github.com/cloudflare/pingora/blob/main/pingora-proxy/examples/gateway.rs

Notice you need prometheus, clap, and log

cargo add clap log prometheus

cargo check

Replace all instances of pingora_{crate} with the appropriate pingora::. This gets you a main.rs that contains:

use async_trait::async_trait;
use clap::Parser;
use log::info;
use pingora::{http::ResponseHeader, prelude::*};
use prometheus::register_int_counter;

fn check_login(req: &RequestHeader) -> bool {
    // implement you logic check logic here
    req.headers.get("Authorization").map(|v| v.as_bytes()) == Some(b"password")
}

pub struct MyGateway {
    req_metric: prometheus::IntCounter,
}

#[async_trait]
impl ProxyHttp for MyGateway {
    type CTX = ();
    fn new_ctx(&self) -> Self::CTX {}

    async fn request_filter(&self, session: &mut Session, _ctx: &mut Self::CTX) -> Result<bool> {
        if session.req_header().uri.path().starts_with("/login")
            && !check_login(session.req_header())
        {
            let _ = session.respond_error(403).await;
            // true: early return as the response is already written
            return Ok(true);
        }
        Ok(false)
    }

    async fn upstream_peer(
        &self,
        session: &mut Session,
        _ctx: &mut Self::CTX,
    ) -> Result<Box<HttpPeer>> {
        let addr = if session.req_header().uri.path().starts_with("/family") {
            ("1.0.0.1", 443)
        } else {
            ("1.1.1.1", 443)
        };

        info!("connecting to {addr:?}");

        let peer = Box::new(HttpPeer::new(addr, true, "one.one.one.one".to_string()));
        Ok(peer)
    }

    async fn response_filter(
        &self,
        _session: &mut Session,
        upstream_response: &mut ResponseHeader,
        _ctx: &mut Self::CTX,
    ) -> Result<()>
    where
        Self::CTX: Send + Sync,
    {
        // replace existing header if any
        upstream_response
            .insert_header("Server", "MyGateway")
            .unwrap();
        // because we don't support h3
        upstream_response.remove_header("alt-svc");

        Ok(())
    }

    async fn logging(
        &self,
        session: &mut Session,
        _e: Option<&pingora::Error>,
        ctx: &mut Self::CTX,
    ) {
        let response_code = session
            .response_written()
            .map_or(0, |resp| resp.status.as_u16());
        info!(
            "{} response code: {response_code}",
            self.request_summary(session, ctx)
        );

        self.req_metric.inc();
    }
}

fn main() {
    env_logger::init();

    // read command line arguments
    let opt = Opt::parse();
    let mut my_server = Server::new(Some(opt)).unwrap();
    my_server.bootstrap();

    let mut my_proxy = pingora::proxy::http_proxy_service(
        &my_server.configuration,
        MyGateway {
            req_metric: register_int_counter!("req_counter", "Number of requests").unwrap(),
        },
    );
    my_proxy.add_tcp("0.0.0.0:6191");
    my_server.add_service(my_proxy);

    let mut prometheus_service_http =
        pingora::services::listening::Service::prometheus_http_service();
    prometheus_service_http.add_tcp("127.0.0.1:6192");
    my_server.add_service(prometheus_service_http);

    my_server.run_forever();
}

Now, Opt::parse() doesn't work. According to RA there may be an alternative option here:

help: trait `Parser` which provides `parse` is implemented but not in scope; perhaps you want to import it
    |
1   + use clap::derive::Parser;
    |
help: there is an associated function `try_parse` with a similar name
    |
90  |     let opt = Opt::try_parse();
    |                    ~~~~~~~~~

No combination of clap traits can work. Discover that pingora::prelude::Opt has an associated parse_args() method. After changing let opt = Opt::parse(); to let opt = Opt::parse_args(); we can compile correctly.

Run

cargo run

Now test it

# returns  Couldn't connect to server
curl 127.0.0.1:6190/family/ -H "Host: one.one.one.one"

# returns bad gateway
curl 127.0.0.1:6191/login/ -H "Host: one.one.one.one" -I -H "Authorization: password"

# returns error as expected
curl 127.0.0.1:6191/login/ -H "Host: one.one.one.one" -I -H "Authorization: bad"

From a browser navigate to localhost:6190/family and expect to be proxied to https://one.one.one.one/

Expected results

I expected the proxy to successfully work.

Observed results

Each example fails to with a 502 or similar error including the quick start.

Additional context

At the 0.1.0 release, the quickstart and the load balancer examples worked. They no longer do.

[momirza]

Adding the openssl feature should fix this issue:

pingora = { version = "0.4", features = ["openssl"] }

[darth-raijin]

+1, furthermore the Proxy example also attempts to start the load-balancer example