Support browser specific protocols (chrome extensions)

[kuceb]

Support cy.visit() with protocols other than http/https such as:

• chrome://
• chrome-extension://
• resource://

Most notably, chrome-extension:// will allow users to test the UI of a chrome extension.

[JulianG]

There's also Add support for testing chrome extensions but it's closed. I've added a comment because I believe the issue is still happening.

[kuceb]

@JulianG that issue was for testing content scripts, which is currently possible. But yeah this is the other half of Chrome extensions

[pureooze]

Would love to see support for this added!

[timrogers]

I’d be very keen to see this - you can already make a fair bit of headway with extensions, but this would be the icing on the cake.

[vborshchov]

I would love to see this ability. It would be so great.

[kuceb]

Curious, which pages are you most wanting to test? background.html? popup.html? And you need access to the chrome.* apis correct? I'm trying to figure out the scope of work here, because embedding the page in an iframe while maintaining access to chrome.* privileged apis might be tricky

[Kocal]

@Bkucera For my own use case, I would be able to run Cypress on my extension's popup in order to see if:

• it actually renders
• some dynamic elements are present
• some links are present too
• internal tabs navigation (vue-router) works

I tried cy.visit('chrome-extension://.../popup.html') but it doesn't work due to invalid protocol.

Also, background.html is literally never used because it's a auto-generated page when you use a single background.js (probably ~99% of the time), but it would be nice for running Cypress on options.html page. :+1:

Having access to chrome* API would be awesome (e.g.: to check if a notification has been triggered, sync, ...), or maybe we can manually stub chrome* methods at the moment?

Thanks you for taking time on this. :slightly_smiling_face:

[JulianG]

@Bkucera Exactly. My chrome extension uses chrome.windows.* and chrome.tabs.* and in order to display in a list and manipulate well... windows and tabs. I'm currently in the process of writing a fake (as in not a mock nor stub) chrome api so I can test in Cypress, but I would very much prefer to test the real thing.

(In my case it's neither background.html nor popup.html. My background.js uses chrome api to open a window containing index.html.)

[pureooze]

In my case I want to test via cy.visit('chrome-extension://.../popup.html') or if possible by using the keyboard hotkeys to trigger the extension. In my case the hotkey causes the background.js to run a script that attaches the HTML for the extension to the DOM. My extension is used to do tab switching so the user could switch to any tab they have open. I also use the history and sessions APIs.

One thing I had hoped to do was call cy.visit a couple times to get cypress to load some browsing history and then run my extension to test the calls to the APIs but I would be willing to settle for stubbing their responses in cypress.

[kuceb]

Thanks everyone for the input. I think the use case of @Kocal can currently be done by stubbing chrome.* apis and running a simple http server in the folder with manifest.json. Then you can just cy.visit http://localhost:8080/popup.html and this should be very similar to visiting chrome-extension://... I have done this before and you can change the viewport to be something more similar to popup size

[Kocal]

My popup is now successfuly tested, thanks you all! :sunglasses:

Some tips for people who wants to spec their extension's views and mocking chrome* API:

• Build your extension (e.g.: output in dist folder)
• Run a local server pointing on your dist folder (serve is pretty nice)
• Update your Cypress configuration:

{
  "baseUrl": "http://localhost:5000"
}

• Into your spec:

  describe('App', () => {
  before(() => {
  
  // Load your popup
  cy.visit('/popup/popup.html', {
  
    // If you need to stub `chrome*` API, you should do it there:
    onBeforeLoad(win) {
      win.chrome = win.chrome || {};
      win.chrome.runtime = {
        sendMessage(message, cb) {
          // ...
          cb(some_data);
        },
      };
    },
  });
  });
  });

[JulianG]

Have you stubbed chrome. in the end? I started making a fake implementation of `chrome.windows.andchrome tabs.*` which is what I use. Do you know if such thing already exists?

[Kocal]

Well, I only use chrome.runtime.sendMessage inside my popup view, I didn't take time for other methods :confused:

EDIT: some times ago I used jest-webextension-mock for mocking chrome.* in Jest context. Maybe it can helps you

[pureooze]

I have used sinon-chrome before, it works well for unit/IT testing. Not sure how it compares to jest-webextension-mock.

[Moejoe90]

For me, the use case would not be testing an extension but testing our app with an extension I want to be able to click some buttons on the extension etc while navigating through our app

[callmenick]

I can see both use cases being strong.

Say you're developing an extension that has some UI flows in the popup, it gets pretty annoying to manually test those each time by clicking the extension icon! Regressions are so hard to notice because of the nature of the popup. With Cypress interface, I'd spot the regressions by just running the test command, and notice exactly where it happens.

On the other side, you're obviously developing an extension to enhance a browser -- and websites as a result -- so running an "example site" inside Cypress with the extension loaded could be useful to notice things like what the extension might inject, etc. like what @Moejoe90 is suggesting.

@Bkucera is this being actively worked on? Is there anything the community can do to support?

[kuceb]

@callmenick I'm working on testing a chrome extension that injects html into a third party site, but no Test Runner features are being developed yet.

Most likely I'll end up using something from here to allow the extension to detect content in the AUT (application under test), which cypress puts in an iframe

[callmenick]

Cool, thanks for the update. I'm personally interested in the first point, supporting extension protocols so I can visit chrome-extension://id/index.html and run tests from there. Is there any roadmap for that?

[Kocal]

@callmenick you can check https://github.com/cypress-io/cypress/issues/1965#issuecomment-415796370 I guess

[callmenick]

Yeah that's what we're doing now as well @Kocal. Thanks for that!

[kuceb]

Hey everyone, I was able to set up a good workflow (*see however below**) for testing chrome extensions that inject HTML onto third party sites.

For example, to test an extension that injects HTML inside of Gmail, there were a few things I had to do:

1. Export the gmail webpages that need testing using this chrome extension: Save Page WE (I saved them to cypress/plugins/gmail/, one of my pages is called gmail_home.html)
2. Host the exported webpages with a express server in plugins/index.js, and it needs to have a self-signed SSL cert: this requires launching cypress with sudo due to needing port 443 (google's port)
   plugins/index.js code:

const path = require('path');

module.exports = (on) => {
  on('task', {
    setUrlPath (url) {
      console.log(url)
      filepath = url
      return null
    }
  })
}

let filepath = 'gmail/gmail_home.html'
const express = require('express')

const PORT = 443

const app = express()
const selfSignedHttps = require('self-signed-https')

app.get('/*', (req, res) => {
  return res.sendFile(filepath, { root: __dirname })  
})

selfSignedHttps(app).listen(PORT, () => console.log(`Server running on port ${PORT}`))

...

3. add to cypress.json:

   "hosts": {
   "mail.google.com": "127.0.0.1",
   }

4. add "all_frames":true to extension's manifest.json

   "content_scripts": [
   {
     "matches": [
       "http://mail.google.com/*",
       "https://mail.google.com/*"
     ],
     "js": [
       ...
     ],
     "css": [
       ...
     ],
     "all_frames": true
   }

5. Finally, load the extension into Cypress by modifying the plugins/index.js file as shown here: https://docs.cypress.io/api/plugins/browser-launch-api.html#Usage

:+1:

However:

Cypress cy.route stubbing will not affect traffic originating in a chrome extension, so there is more work to do before that can be used.

This will be fixed by Full Network stubbing, and is on our roadmap: #687

[whymarrh]

Similar to @callmenick above, I'm curious about whether or not Cypress can support simply visit a HTML page from a Chrome extension. If an extension has an HTML page listed in its web_accessible_resources it can be loaded in an iframe without too much trouble. That manifest entry, coupled with the --load-extension=$folder flag could allow a very basic UI test of a Chrome extension. Is this something Cypress could support?

[bdresser]

@Bkucera any idea on when this might happen? Asking since it'll influence our testing roadmap.

Thanks for all your hard work :)

[kuceb]

@bdresser the main hurdle to this is #687 , and I have no timeline for that; however I'm not exactly sure what you are testing. could you describe your use case like others have above?

[karlbecker]

Thanks for the information thread and guidance, @Bkucera , along with the example!

To fully test my Chrome extension, I want to be mocking out a variety of API responses, including 2xx, 4xx, and 5xx responses from my API. If I understand correctly, I cannot currently do this with Cypress by itself. There is also no way you can easily think of to use some other library to mock it out that will work with running tests in Cypress, correct?

I think I can accomplish this by using Puppeteer directly, as described in this SO answer and using nock to mock the API responses.

[LyubovKorovina]

Is chrome-extension protocol working? It fails with error > Error: Invalid protocol: chrome-extension:

[geekyed]

I also have a use case where I would like to use Cypress, but looking at this issue, wont work. I would like to test a chrome extensions UI that communicates to a webpage using chrome.runtime.connect(extID, portName). The extension has a state stored in redux using webext-redux to share the store, which I think needs to run from within an extension and wont work by hosting the files locally. Being able to test pages starting with chrome-extension:// would be amazing.

[kepi]

I belive there should be way to test extension in real Chrome. Not under web server, not stubbing anything. With doing all this extra work, there is huge space to introduce new bugs and worse - to miss some real bugs.

I didn't check the source code of Cypress or Chrome so I'm really not sure how it is working under the hood. But isn't it possible to navigate to chrome:// and chrome-extension:// in exact same way as to http://? Isn't it only alias to http and working over http protocol?

Btw. I started to play with Cypress only because I need to test extension, I wouldn't give it a try if it wouldn't look like it can do this from docs. As other tools are lacking in this area too, it my be great to provide better support for extension authors.

[stoically]

@Bkucera

the main hurdle to this is #687 , and I have no timeline for that; however I'm not exactly sure what you are testing. could you describe your use case like others have above?

Since #687 landed in the meantime, would you say there are other major blockers? I'd be interested in working on making this happen and some rough pointers would be appreciated.

[jennifer-shehane]

https://github.com/cypress-io/cypress/issues/687 is actually not a prerequisite to delivering this feature. There was a misunderstanding previously on the scope of that issue.

[stoically]

@jennifer-shehane Thanks for the info. Do you by chance know if there's something important missing currently which would make implementing this feature especially hard? Asking because it's tagged with "difficulty 5".

[George-cl]

As of now is there a robust and reliable method for testing a chrome extension - i.e. is it possible for Cypress to open the extension window/popup and interact with the UI? I would require a test to initially start on a website and in the course of testing, it will send requests and receive responses from the extension based on user input on both the site and extension.

Example scenario: Site: user clicks on a button Extn: receives request from site User: Navigates to extension to confirm/deny request Extn: returns response based on user input Site: handles response

[nidasoyab]

My popup is now successfuly tested, thanks you all! 😎

Some tips for people who wants to spec their extension's views and mocking chrome* API:

• Build your extension (e.g.: output in dist folder)
• Run a local server pointing on your dist folder (serve is pretty nice)
• Update your Cypress configuration:

{
  "baseUrl": "http://localhost:5000"
}

• Into your spec:

describe('App', () => {
  before(() => {

    // Load your popup
    cy.visit('/popup/popup.html', {

      // If you need to stub `chrome*` API, you should do it there:
      onBeforeLoad(win) {
        win.chrome = win.chrome || {};
        win.chrome.runtime = {
          sendMessage(message, cb) {
            // ...
            cb(some_data);
          },
        };
      },
    });
  });
});

can you share the repository name of this

[erfanatp]

My popup is now successfuly tested, thanks you all! 😎 Some tips for people who wants to spec their extension's views and mocking chrome* API:

• Build your extension (e.g.: output in dist folder)
• Run a local server pointing on your dist folder (serve is pretty nice)
• Update your Cypress configuration:

{
  "baseUrl": "http://localhost:5000"
}

• Into your spec:

describe('App', () => {
  before(() => {

    // Load your popup
    cy.visit('/popup/popup.html', {

      // If you need to stub `chrome*` API, you should do it there:
      onBeforeLoad(win) {
        win.chrome = win.chrome || {};
        win.chrome.runtime = {
          sendMessage(message, cb) {
            // ...
            cb(some_data);
          },
        };
      },
    });
  });
});

can you share the repository name of this

Serving doesn't work. The browser says "local" and some specific Chrome extension APIs are undefined...

[juandavidkincaid]

For those who are still looking for alternatives or ideas... Try binding puppeteer on a CDP session from the plugins scope, it will enable you to access the chrome-extension:// protocol, something similar to what synpress does, a little bit of work, but completely resolves the issue, at least for now.

Something like this...

const { data: debuggerInfo } = await axios.get(
  `http://localhost:${this.config.debugPort}/json/version`
)

this.browser = await puppeteer.connect({
  browserWSEndpoint: debuggerInfo.webSocketDebuggerUrl,
  ignoreHTTPSErrors: true,
  defaultViewport: null,
})

Then you would access extension pages from puppeteer instead of cypress, do clicks and stuff, but still using and testing with cypress.

[shaun-wild]

Is there any news on this issue?

Really would like to be able to test my extension.

[salimdriai]

My popup is now successfuly tested, thanks you all! 😎

Some tips for people who wants to spec their extension's views and mocking chrome* API:

• Build your extension (e.g.: output in dist folder)
• Run a local server pointing on your dist folder (serve is pretty nice)
• Update your Cypress configuration:

{
  "baseUrl": "http://localhost:5000"
}

• Into your spec:

describe('App', () => {
  before(() => {

    // Load your popup
    cy.visit('/popup/popup.html', {

      // If you need to stub `chrome*` API, you should do it there:
      onBeforeLoad(win) {
        win.chrome = win.chrome || {};
        win.chrome.runtime = {
          sendMessage(message, cb) {
            // ...
            cb(some_data);
          },
        };
      },
    });
  });
});

can someone please explain how to build the extension and point server to it's dist folder

[salimdriai]

For those who are still looking for alternatives or ideas... Try binding puppeteer on a CDP session from the plugins scope, it will enable you to access the chrome-extension:// protocol, something similar to what synpress does, a little bit of work, but completely resolves the issue, at least for now.

Something like this...

const { data: debuggerInfo } = await axios.get(
  `http://localhost:${this.config.debugPort}/json/version`
)

this.browser = await puppeteer.connect({
  browserWSEndpoint: debuggerInfo.webSocketDebuggerUrl,
  ignoreHTTPSErrors: true,
  defaultViewport: null,
})

Then you would access extension pages from puppeteer instead of cypress, do clicks and stuff, but still using and testing with cypress.

is there any guide to implement this ?

[shaun-wild]

I've switched over to Nightwatch for now.

[Depetrol]

same question as @iskandar47 , any specific guidelines? The introduction is kinda vague and cannot replicate

[juandavidkincaid]

Sorry for the late response @iskandar47 and @Depetrol... The snippet shared corresponds to an issue related to CDP sessions in synpress... The actual implementation is better understood in synpress code at the moment I don't have enough time to do a demo but is basically the same... The solution uses cypress plugins and puppeteer, cypress reveals a WS socket for debugging purposes and the solution then connects that WS endpoint to puppeteer to control the browser in puppeteer side...

[Depetrol]

Our team have successfully loaded the browser extension page in cypress:

The reason why chrome-extension:// is blocked: cypress runs in-browser and is blocked by chrome security policies

Solution: use puppeteer to control the browser behavior externally with CDP (Chrome Devtools Protocol), bypassing the security limitations.

Here's how to do it: Load the following code in cypress/plugins/index.js

const axios = require('axios');
const puppeteer = require('puppeteer');
const path = require('path');

const buildDir = path.join(__dirname, '../../build');
const extensionURL = // this is fixed for our extension
  'chrome-extension://abcdefdsbfdsbafdbsfbabsffdbsfbds/index.html';

let debuggingPort = null;

async function setBrowser() {
  if (debuggingPort != null) {
    const url = `http://127.0.0.1:${debuggingPort}/json/version`;
    const { data: debuggerInfo } = await axios.get(url);
    const browser = await puppeteer.connect({
      browserWSEndpoint: debuggerInfo.webSocketDebuggerUrl,
      ignoreHTTPSErrors: true,
      defaultViewport: null,
    });
    const openTabs = await browser.targets();
    const globalPage = await openTabs
      .find(p => p._targetInfo.type === 'page')
      .page();
    const iframeElement = await globalPage.$('iframe.aut-iframe');
    const theFrame = await iframeElement.contentFrame();
    await theFrame.goto(extensionURL);
  } else {
    throw new Error('debuggingPort not provided');
  }

  return null;
}

module.exports = (on, config) => {
  on(
    'before:browser:launch',
    async (browser = { headless: true }, launchOptions) => {
      if (browser.name !== 'electron' || browser.name === 'chromium') {
        // Here you will need to persist the port to your plugins, whatever they may be
        const RDP = launchOptions.args.find(
          arg => arg.slice(0, 23) === '--remote-debugging-port',
        );
        debuggingPort = RDP.split('=')[1];

        launchOptions.extensions.push(buildDir);
      }

      return launchOptions;
    },
  );

  on('task', {
    setBrowser,
  });
};

run the following code before each test

describe('My First Test', () => {
  it('Visits a chrome-extension', () => {
    cy.task('setBrowser')
      .get('.btn-primary')
      .click()
      .children()
      .then(e => console.info(e));
  });
});

[Tofel]

@Depetrol were you able to run these tests in the CI? For me everything works on local (in a fully dockerised setup), but when running in Github Actions it seems that the extension never gets loaded (or at least Puppeteer doesn't find the second tab with the extension within 60 seconds; it's hard to say what's really happening, because Cypress` video/screenshot only includes browser's viewport).

[escapedcat]

@Depetrol cool! This looks like what other people try to do with Playwright as well. Here's an example on how to dynamically get the extensionId: https://github.com/xcv58/Tab-Manager-v2/blob/master/packages/integration_test/util.ts#L51-L55

I recently tried Playwright/Puppeteer and Cypress/Puppeteer and both was working ok. Biggest issue I've noticed for both approaches was that neither could provide the screenshots/videos on fail or or a nice trace of commands. I beleive this happens because Puppeteer is taking over. Not sure though.

Did you get i.e. screenshots on fail or the commands-trace with your Cypress approach? Is that's what the debgging-port is about?

[will-beta]

@Depetrol were you able to run these tests in the CI? For me everything works on local (in a fully dockerised setup), but when running in Github Actions it seems that the extension never gets loaded (or at least Puppeteer doesn't find the second tab with the extension within 60 seconds; it's hard to say what's really happening, because Cypress` video/screenshot only includes browser's viewport).

@Tofel Yes. Try to install a new version of Chrome.

[will-beta]

@Depetrol cool! This looks like what other people try to do with Playwright as well. Here's an example on how to dynamically get the extensionId: https://github.com/xcv58/Tab-Manager-v2/blob/master/packages/integration_test/util.ts#L51-L55

I recently tried Playwright/Puppeteer and Cypress/Puppeteer and both was working ok. Biggest issue I've noticed for both approaches was that neither could provide the screenshots/videos on fail or or a nice trace of commands. I beleive this happens because Puppeteer is taking over. Not sure though.

Did you get i.e. screenshots on fail or the commands-trace with your Cypress approach? Is that's what the debgging-port is about?

@escapedcat Yes. Cypress will automatically capture screenshots when a failure happens during cypress run.

[escapedcat]

@escapedcat Yes. Cypress will automatically capture screenshots when a failure happens during cypress run.

Yeah, that's the default, right? When I tested it using puppeteer (to test an extension) it didn't. It also didn't show a nice command path in the left panel like I'm used to.

Might need to try again.

[Tofel]

No it's not ;-) Cypress knows nothing about Puppeteer, which is used to communicate with MetaMask, so it will never take the screenshot of the extension (regardless whether it's the separate tab it's running in or the notification).

[Tofel]

@Depetrol were you able to run these tests in the CI? For me everything works on local (in a fully dockerised setup), but when running in Github Actions it seems that the extension never gets loaded (or at least Puppeteer doesn't find the second tab with the extension within 60 seconds; it's hard to say what's really happening, because Cypress` video/screenshot only includes browser's viewport).

@Tofel Yes. Try to install a new version of Chrome.

I wished it was that easy, Chrome 101 has memory leaks, which makes it unusable :/

[escapedcat]

No it's not ;-) Cypress knows nothing about Puppeteer, which is used to communicate with MetaMask, so it will never take the screenshot of the extension (regardless whether it's the separate tab it's running in or the notification).

@Tofel right, I think that's what i meant. If you use Cypress only per default Cypress will do all these things but once you handle everything with Puppeteer Cypress doesn't get these infos anymore and won't create i.e. screenshots on fail.