Intercepting/testing network requests

[Danack]

Hi,

quick question - does this library support setting up intercepting, or otherwise debugging, network requests?

Basically, I'm trying to write an integration test to check that our content security policy headers are correctly blocks some external requests, so I'd like to check that:

• calls to unapproved Javascript files don't succeed.
• after that request is blocked, there is an appropriate call to our content security policy reporting end-point.

cheers Dan

[jakubkulhan]

Hello!

Yes, you can register listeners to be notified about network requests/responses like this:

use ChromeDevtoolsProtocol\Instance\Launcher;
use ChromeDevtoolsProtocol\Model\Network\EnableRequest;
use ChromeDevtoolsProtocol\Model\Network\LoadingFailedEvent;
use ChromeDevtoolsProtocol\Model\Network\LoadingFinishedEvent;
use ChromeDevtoolsProtocol\Model\Network\RequestWillBeSentEvent;
use ChromeDevtoolsProtocol\Model\Network\ResponseReceivedEvent;
use ChromeDevtoolsProtocol\Model\Page\NavigateRequest;

$ctx = Context::withTimeout(Context::background(), 10);
$launcher = new Launcher();
$instance = $launcher->launch($ctx);
try {
    $session = $instance->createSession($ctx);
    try {
        $session->page()->enable($ctx);
        $session->network()->enable($ctx, EnableRequest::builder()->build());

        $session->network()->addRequestWillBeSentListener(function (RequestWillBeSentEvent $ev) {
            // ...
        });
        $session->network()->addResponseReceivedListener(function (ResponseReceivedEvent $ev) {
            // ...
        });
        $session->network()->addLoadingFailedListener(function (LoadingFailedEvent $ev) {
            // ...
        });
        $session->network()->addLoadingFinishedListener(function (LoadingFinishedEvent $ev) {
            // ...
        });

        $session->page()->navigate(
            $ctx,
            NavigateRequest::builder()
                ->setUrl("...")
                ->build()
        );

        $session->page()->awaitLoadEventFired($ctx);

        // ...

    } finally {
        $session->close();
    }
} finally {
    $instance->close();
}

Documentation for Network domain of DevTools Protocol is available at https://chromedevtools.github.io/devtools-protocol/tot/Network

[Danack]

Thanks for that.....I think possibly I might need to make a reproducible example, but I'm seeing one weird thing.

It looks like the response headers aren't being populated from the response. From the debug window of PhpStorm for the response of the page:

headers = {ChromeDevtoolsProtocol\Model\Network\Headers} [0]

headersText = "HTTP/1.1 200 OK\r\nServer: nginx/1.10.3\r\nDate: Mon, 22 Jan 2018 12:08:47 GMT\r\nContent-Type: text/html;charset=UTF-8\r\nTransfer-Encoding: chunked\r\nConnection: keep-alive\r\nVary: Accept-Encoding\r\nContent-Security-Policy: default-src 'self'; script-src 'self' 'nonce-be4e92cbb7f24ef5889059a0'; style-src 'self' 'unsafe-inline'; report-uri /csp\r\nContent-Encoding: gzip\r\n\r\n"

I've got a suspicion that might actually be a bug in Chrome, as there is a request for jquery from that page to load jquery from another page, that should have been blocked by the CSP Content-Security-Policy: default-src 'self'; script-src 'self' 'nonce-be4e92cbb7f24ef5889059a0';

However the response for the jquery scrip has:

url = "https://ajax.googleapis.com/ajax/libs/jquery/3.2.1/jquery.min.js"
status = 200

Exactly as if Chrome hasn't decoded the header string into the individual headers internally...

Just to check, the headers array is normally populated for you when you listen to the addResponseReceivedListener() ?

[jakubkulhan]

You found a bug in the library. ChromeDevtoolsProtocol\Model\Network\Headers objects weren't properly populated from protocol data. I've fixed it in d8b49a3.

[Danack]

Thanks for that, the headers field is populated for me now, using version 0.3.0.

Incidentally though, the headersText that was previously populated now appears to be null.

Also incidentally, it looks like there is a bug in the Chrome headless content security policy. While using your library to tell Chrome headless to visit a page that has a CSP in place of:

default-src 'self'; script-src 'self' 'nonce-deaf66290ab08019c87ff071'; style-src 'self' 'unsafe-inline'; report-uri /csp

Chrome headless launches a request for an external Javascript file that should be blocked by that CSP......which I'll report upstream.

[jakubkulhan]

headersText seems to be populated only when the request is sent over HTTP/1.1. For HTTP/2 requests headersText remains null. It makes sense, because HTTP/2 headers aren't really sent like this.

You may also check Log.entryAdded / Console.messageAdded events to see whether Chrome reports any CSP violations to console.