intoli / remote-browser

A low-level browser automation framework built on top of the Web Extensions API standard.
Other
1.74k stars 105 forks source link

How to intercept requests? #71

Closed Raidus closed 6 years ago

Raidus commented 6 years ago

Hi,

I'm figuring out how to intercept requests. I have to following snippet inside my code:

    browser.webRequest.onBeforeRequest.addListener(
      function(details) {
        return { cancel: details.url.indexOf('news.ycombinator.com') != -1 };
      },
      { urls: ['<all_urls>'] },
      ['blocking'],
    );

Inside the manifest.json (remote-browser module) I've following permissions:

"permissions": ["<all_urls>", "tabs", "webRequest", "webRequestBlocking"]

But my code gives following error:

Invalid listener for webRequest.onBeforeRequest

What am I doing wrong?

Raidus commented 6 years ago

That's the test script

const { Browser } = require('remote-browser');

(async () => {
  // Launch a remote browser instance.
  const browser = new Browser();
  await browser.launch();

  browser.webRequest.onBeforeRequest.addListener(
    function(details) {
      return { cancel: details.url.indexOf('news.ycombinator.com') != -1 };
    },
    { urls: ['<all_urls>'], types: ['main_frame'] },
    ['blocking'],
  );

  // Navigate to Hacker News.
  const tab = await browser.tabs.create({
    url: 'https://news.ycombinator.com',
  });

  // Extract all of the stories on the front page.
  const stories = await browser[tab.id](() =>
    Array.from(document.getElementsByClassName('athing')).map(
      (tr, position) => {
        const storyLink = tr.getElementsByClassName('storylink')[0];
        const score = tr.nextSibling.getElementsByClassName('score')[0];
        return {
          position: position + 1,
          title: storyLink.innerHTML,
          url: storyLink.href,
          score: score && score.innerHTML,
        };
      },
    ),
  );

  console.log('Hackernews found', stories.length);

  await browser.quit();
})();
sangaline commented 6 years ago

Thanks, this is definitely a bug. The browser.webRequest was undefined due to a permissions issue which should now be fixed in v0.0.14. There also seems to be a problem with how the listener function is being serialized, I'll continue to look into this and give an update here once it's been resolved.

sangaline commented 6 years ago

This should be working as of v0.0.15, thanks again for the report. One thing to note is that you should await the browser.webRequest.onBeforeRequest.addListener() call to be 100% sure that it evaluates before the navigation. Another thing is that blocking the request seems to put the tab in a sort of dead state; you can't right click in it, you can't open the developer tools, and no content scripts are evaluated. This manifests as an uninformative timeout error right now when you try to evaluate code in the content script of the blocked page. We should make the error message more informative and recoverable, but the blocking itself is working.

I'm going to close this issue now, but let me know if you're still experiencing any problems and we'll reopen it.