No traffic when using `rustls-tls`

martinmose commented 1 week ago

Description

First and foremost, thank you for an excellent piece of software! My colleagues and I use Postman extensively in our development workflow for native iOS/Android apps. Recently, we've started integrating Rust alongside Swift/Kotlin, primarily using Rust for handling network tasks. For HTTP client functionality, we rely on the popular reqwest crate.

To avoid dealing with OpenSSL, we've switched to using rustls-tls. Our Cargo.toml configuration looks like this:

reqwest = { version = "0.12.4", features = [
  "json",
  "rustls-tls",
], default-features = false }

However, after this change, we no longer see any traffic in Proxyman. We would greatly appreciate it if this could be supported again. It might be an edge case (apologies for that), but I'm posting this issue to see if there's something obvious we've missed.

Steps to Reproduce

main.rs:

use reqwest::Error;

#[tokio::main]
async fn main() -> Result<(), Error> {
    let response = reqwest::get("https://httpbin.org/get").await?;

    if response.status().is_success() {
        let body = response.text().await?;
        println!("Response Text: {}", body);
    } else {
        println!("Request failed with status: {}", response.status());
    }

    Ok(())
}

Cargo.toml:

[package]
name = "simple_request"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = { version = "0.12.5", features = [
  "json",
  "rustls-tls",
], default-features = false }
tokio = { version = "1.38.0", features = ["full"] }

Current Behavior

No traffic appears in Proxyman.

Expected Behavior

When the Cargo.toml is configured like this:

[package]
name = "simple_request"
version = "0.1.0"
edition = "2021"

[dependencies]
reqwest = "0.12.5"
tokio = { version = "1.38.0", features = ["full"] }

We see the request in Proxyman.

Environment

App version: 5.5.0 (50500)
macOS version: Sonoma 14.5

NghiaTranUIT commented 1 week ago

Hi, it's not a bug from Proxyman. It's how the reqwest lib works. By default, the reqwest doesn't use any proxy, so Proxyman can't capture it automatically.

To capture traffic from `reqwest`

You might manually set the HTTP/HTTPS Proxy when initializing the client.
Here is the code: https://docs.rs/reqwest/latest/reqwest/struct.Proxy.html#method.https
it should proxy to http://localhost:9090

To decrypt HTTPS data

By default, the reqwest doesn't trust Proxyman self-signed certificate, so you might get SSL Error when Proxyman tries to decrypt it.
To fix it, here is the code to disable the SSL Verification: https://github.com/seanmonstar/reqwest/issues/182#issuecomment-469997565

If it's from Ruby, Python or NodeJS, Proxyman has the Auto Setup tool, but not for Rust.

I haven't supported it yet, so we have to manually do it.

martinmose commented 1 week ago

Hi,

I wasn't sure how to classify this "issue", so I chose to label it as a bug. 😅

Thank you very much for the helpful information! We can definitely work around that. I can confirm that it works if I do the following:

use anyhow::Result;
use reqwest::Url;

#[tokio::main]
async fn main() -> Result<()> {
    let proxy_url = Url::parse("http://localhost:9090")?;
    let client = reqwest::Client::builder()
        .danger_accept_invalid_certs(true)
        .proxy(reqwest::Proxy::https(proxy_url)?)
        .build()?;
    let response = client.get("https://httpbin.org/get").send().await?;

    if response.status().is_success() {
        let body = response.text().await?;
        println!("Response Text: {}", body);
    } else {
        println!("Request failed with status: {}", response.status());
    }

    Ok(())
}

NghiaTranUIT commented 1 week ago

Thanks, I will update our Documentation to make Proxyman work with Rust + Reqwest 🙇

ProxymanApp / Proxyman