How to capture the Chromium's requests and responses traffic using thirtyfour

[Codekies]

I want to use Rust with thirtyfour to open chromium and capture my HTTPS web app traffic. It is for the security tools development usage.

[stevepryde]

It's technically possible using CDP but you'll need to wire it up yourself.

CDP example here https://github.com/stevepryde/thirtyfour/blob/main/thirtyfour/examples/chrome_devtools.rs

Chrome devtools protocol docs here https://chromedevtools.github.io/devtools-protocol/

[Codekies]

Hi Steve, Thanks a lot for the help. I have written some code to access google.com and capture the HTTP requests, response

use thirtyfour::extensions::cdp::ChromeDevTools;
use thirtyfour::prelude::*;

#[tokio::main]
async fn main() -> WebDriverResult<()> {
    let caps = DesiredCapabilities::chrome();
    let driver = WebDriver::new("http://localhost:9515", caps).await?;

    // Use Chrome Devtools Protocol (CDP).
    let dev_tools = ChromeDevTools::new(driver.handle.clone());

    // Enable network tracking.
    dev_tools.execute_cdp("Network.enable").await?;

    // Navigate to https://www.google.com.
    driver.goto("https://www.google.com").await?;

    // Wait for a short duration to allow requests and responses to be captured.
    tokio::time::sleep(std::time::Duration::from_secs(5)).await;

    // Get the captured network events.
    let request_events = dev_tools.execute_cdp("Network.Request").await?;
    let response_events = dev_tools.execute_cdp("Network.Response").await?;

    // Print the captured network events.
    println!("Requests:");
    println!("{:?}", request_events);

    println!("Responses:");
    println!("{:?}", response_events);

    // Disable network tracking.
    dev_tools.execute_cdp("Network.disable").await?;

    // Always explicitly close the browser. There are no async destructors.
    driver.quit().await?;

    Ok(())
}

but when I ran the executable, it showed me this. Error: UnknownCommand(WebDriverErrorInfo { status: 404, error: "", value: WebDriverErrorValue { message: "unknown command: 'Network.Request' wasn't found\n (Session info: chrome=123.0.6312.123)", error: Some("unknown command"), stacktrace: Some("\tGetHandleVerifier [0x00007FF7F97A7072+63090]\n\t(No symbol) [0x00007FF7F9712CC2]\n\t(No symbol) [0x00007FF7F95AEC65]\n\t(No symbol) [0x00007FF7F959C96F]\n\t(No symbol) [0x00007FF7F959B100]\n\t(No symbol) [0x00007FF7F959BA8F]\n\t(No symbol) [0x00007FF7F959B9C0]\n\t(No symbol) [0x00007FF7F95B1973]\n\t(No symbol) [0x00007FF7F963FEBA]\n\t(No symbol) [0x00007FF7F9616FDA]\n\t(No symbol) [0x00007FF7F9633412]\n\t(No symbol) [0x00007FF7F9616D83]\n\t(No symbol) [0x00007FF7F95E83A8]\n\t(No symbol) [0x00007FF7F95E9441]\n\tGetHandleVerifier [0x00007FF7F9BA25CD+4238285]\n\tGetHandleVerifier [0x00007FF7F9BDF72D+4488493]\n\tGetHandleVerifier [0x00007FF7F9BD7A0F+4456463]\n\tGetHandleVerifier [0x00007FF7F98805B6+953270]\n\t(No symbol) [0x00007FF7F971E58F]\n\t(No symbol) [0x00007FF7F9719264]\n\t(No symbol) [0x00007FF7F971939B]\n\t(No symbol) [0x00007FF7F9709BD4]\n\tBaseThreadInitThunk [0x00007FFD482A257D+29]\n\tRtlUserThreadStart [0x00007FFD48EAAA48+40]\n"), data: None } }) Would you mind helping me to write an example code if you have time? Thanks a lot.

[stevepryde]

Hmm, looking at the docs here: https://chromedevtools.github.io/devtools-protocol/tot/Network/

It looks like Network.Request is a type, not a command.

I've seen some docs pointing at setRequestInterception: https://chromedevtools.github.io/devtools-protocol/tot/Network/#method-setRequestInterception but that shows as deprecated. It says to try the Fetch domain but I couldn't see anything obvious there.

There is some info here that might be useful: https://www.selenium.dev/documentation/webdriver/bidirectional/chrome_devtools/

I'm going to take a look at the selenium bindings and see how they do it there.

[stevepryde]

There's more info in #55 that might be helpful. Also check out https://github.com/mattsse/chromiumoxide which uses CDP directly without the webdriver side (more like puppeteer).

[stevepryde]

Also https://github.com/btbaggin/cdp-rs