segment-boneyard / nightmare

A high-level browser automation library.
https://open.segment.com
19.55k stars 1.08k forks source link

Blocking NightmareJS from requesting files by regex #1117

Closed elderbas closed 6 years ago

elderbas commented 7 years ago

How can I prevent NightmareJS from making requests to certain resources by way of regex or something similar?

I'm already using this config to prevent images from being requested, but what if I want to block font, css or other resources from being requested?

const nightmare = Nightmare({
  webPreferences: { images: false },
});
casesandberg commented 7 years ago

There currently isn't a way of doing this, will add it as a feature request.

TimNZ commented 7 years ago

Could perhaps be supported via this, passing in an array of strings/regex to Nightmare options.

This look right to you?

https://github.com/electron/electron/blob/master/docs/api/web-request.md#webrequestonbeforerequestfilter-listener

matthewmueller commented 6 years ago

hi @elderbas, while this isn't available in core, it's pretty easy to add a custom action to nightmare to support this. Here's an example of how to remove all the images from google.com:

var Nightmare = require('nightmare')

main().catch(console.error)

async function main() {
  // define a new action
  Nightmare.action(
    'onBeforeRequest',
    //define the action to run inside Electron
    function(name, options, parent, win, renderer, done) {
      win.webContents.session.webRequest.onBeforeRequest((details, cb) => {
        // call our event handler
        parent.call('onBeforeRequest', details, res => {
          res ? cb(Object.assign({}, res)) : cb({ cancel: false })
        })
      })
      done()
    },
    function(handler, done) {
      // listen for "onBeforeRequest" events
      this.child.respondTo('onBeforeRequest', handler)
      done()
    }
  )

  const nightmare = Nightmare({ show: true })

  // start listening
  await nightmare.onBeforeRequest((details, cb) => {
    if (details.resourceType === 'image') {
      return cb({ cancel: true })
    }
    cb({ cancel: false })
  })

  await nightmare.goto('https://google.com')

  await nightmare.end()
}

function sleep(ms) {
  return new Promise(res => setTimeout(res, ms))
}