Monitor network requests

[bitflip2]

Is there a way to subscribe to or wait for specific network requests (for example all requests to a specified domain) and retrieve all transmitted parameters?

[timsuchanek]

@nano2K Thanks for asking this, currently chromeless doesn't expose this functionality, but the chrome debugging interface actually makes it possible. What is your use case? We definitely could add something like this in the future

[bitflip2]

In my case I'd like to use chromeless to test and verify our integration of javascript-tags like Google Analytics, but I am pretty sure such a feature would be great for testing other types of APIs too.

[bitflip2]

I'm currently trying to prepare a PR for this issue, but I'm not sure what's the best approach:

Extending src/chrome/local-runtime.ts in the following way would be the easiest but also the most naive solution I guess:

private async goto(url: string, onRequest: Function): Promise<void> {
  const {Network, Page} = this.client
  await Promise.all([Network.enable(), Page.enable()])
  await Network.setUserAgentOverride({userAgent: `Chromeless ${version}`})
  //---
  if(onRequest) {
    this.log(`logging network requests`)
    Network.requestWillBeSent(onRequest)
  }
  //---
  await Page.navigate({url})
  await Page.loadEventFired()
  this.log(`Navigated to ${url}`)
}

This is how the usage would look like:

const { Chromeless } = require('chromeless')

async function run() {
  const chromeless = new Chromeless()

  const onRequest = (params) => {
    if (params.request.url.includes('my.domain.xyz/api/something')) {
      console.log(params.request);
    }
  }

  const screenshot = await chromeless
    .goto('https://my.domain.xyz', onRequest)
    .wait('#resultStats')
    .screenshot()

  console.log(screenshot) // prints local file path or S3 url

  await chromeless.end()
}

run().catch(console.error.bind(console))

The problem here is that there is no way to use the promise chain and wait for a request either invoked via a third-party-javascript loaded by the page like in my use case or for example when clicking on a dom-element such as a 'search'-button.

Does this make any sense? :smile:

[skunkwerk]

My use case is slightly different - I'd like to just be able to see every network request, like you can in dev tools. URL, method, POST data, response, etc.

[redsquare]

It would be handy to be able to blacklist urls so that our headless synthetic checks do not create analytic requests etc, newRelic have a feature where you can addHostnameToBlacklist

[dowsanjack]

So my use case is, I would like to record and stub HTTP or Socket responses through the code, is that something we can do with this pull request?