Add support for testing Electron.js applications

[bahmutov]

Currently Cypress can open the browser and load websites for testing. Electron.js applications ARE in essence the browser, so it would be cool to let Cypress test them.

Original issue: https://github.com/cypress-io/cypress/issues/2072

• [x] create basic Electron.js example to be used as a proof of concept
  • see https://github.com/cypress-io/cypress-example-electron
• [x] figure out how to change the browser launch from Cypress in order to "attach" to an Electron application
  • external process, similar to Chrome, passing remote interface url to be able to control the browser process
• [ ] see if we can implement testing an Electron.js application with minimal core changes, and most of the code as external plugin (faster iteration)
• [ ] figure out the changes / API that a typical Electron.js application would need to be testable from Cypress

[bahmutov]

Hmm, just trying to load our extensions into Electrons browser window

if (args['--load-extension']) {
    const extensions = args['--load-extension'].split(',')
    extensions.forEach((ext) => {
      console.log('loading extension', ext)
      const name = BrowserWindow.addExtension(ext)
      console.log('extension has returned name: %s', name)
    })
    console.log('loaded extensions\n%s', extensions.join('\n'))
  }
  // Create the browser window.
  mainWindow = new BrowserWindow({
    width: 800,
    height: 600,
    webPreferences: {
      // ? should we just preload Cypress scripts if passed
      preload: path.join(__dirname, 'preload.js'),
      nodeIntegration: false,
      nativeWindowOpen: true,
      webSecurity: false,
      devTools: true,
    }
  })

At first, seems ok

loading extension /Users/gleb/Library/Application Support/Cypress/cy/development/browsers/cypress-example-electron/interactive/CypressExtension
extension has returned name: Cypress
loading extension /Users/gleb/git/cypress/packages/extension/theme
extension has returned name: Cypress Theme
loaded extensions
/Users/gleb/Library/Application Support/Cypress/cy/development/browsers/cypress-example-electron/interactive/CypressExtension
/Users/gleb/git/cypress/packages/extension/theme

but then seems to give errors:

[222:0812/112909.516074:ERROR:CONSOLE(7946)] "Skipping extension with invalid URL: chrome-extension://cypress", source: devtools://devtools/bundled/shell.js (7946)
[222:0812/112909.516118:ERROR:CONSOLE(7946)] "Skipping extension with invalid URL: chrome-extension://cypress-theme", source: devtools://devtools/bundled/shell.js (7946)
GET /__cypress/runner/cypress_runner.js 200 2.142 ms - -

[brian-mann]

You can use the --remote-debugging-port flag documented here: https://electronjs.org/docs/all#supported-chrome-command-line-switches

Put it in environment.coffee and you're good to go. You'll connect to the browser window instance the same way / same logic as we do for chrome RDP.

[bahmutov]

Ok, did a test with Electron - it seems the --remote-debugging-port option is passed directly to the main window, which allows one to connect directly via debugging protocol. The debugging commands are shown below

{
  "scripts": {
    "start": "electron .",
    "debug-main-process": "electron --inspect=5858 .",
    "debug-window-process": "electron --remote-debugging-port=9222 ."
  }
}

The output from both (one for Node debugging, another for the main window browser debugging)

$ npm run debug-main-process

> electron-quick-start@1.0.0 debug-main-process /Users/gleb/git/cypress-example-electron
> electron --inspect=5858 .

Debugger listening on ws://127.0.0.1:5858/8ab0188e-c271-4932-b2f0-f4bb4484dded
For help, see: https://nodejs.org/en/docs/inspector
in /Users/gleb/git/cypress-example-electron/main.js
~/git/cypress-example-electron on master*
$ npm run debug-window-process

> electron-quick-start@1.0.0 debug-window-process /Users/gleb/git/cypress-example-electron
> electron --remote-debugging-port=9222 .

in /Users/gleb/git/cypress-example-electron/main.js

DevTools listening on ws://127.0.0.1:9222/devtools/browser/b2d8ee31-3ad2-4daf-ba29-b624a3b3033a

So Cypress maybe will be able to control external Electron application's main browser window via the remote interface. Luckily we are going this way in this PR https://github.com/cypress-io/cypress/pull/4628

[bahmutov]

Nice, using code from add-cri-4608 I am able to grab the remote interface and control the main window

The switcher for the browser / app

[bahmutov]

Added controlling the external Electronjs main window through chrome remote interface, allowing the main window to show and run tests

Next question - what would the test actually do? For the example application, here is its "normal" behavior

I would like to have the test be:

cy.visit() // without anything, should load main.js
cy.get('#node-version').should('equal', '12.4.0')

[bahmutov]

Hacking around to get two windows going - one with specs, another the real Elecron app window.

And now need to figure out how to use that second application window as the test window (instead of the app iframe)

[bahmutov]

In my example electron app, the window under test is loading the index.html directly, which I think might be a common cast. It has document.domain = '' and so the test window has to get the same domain. I have managed to load the plain file in the test window and from the console am able to access the main window to get the element (even if the element is incorrectly shown!)

If only I could get the script running in the test window, then all would be perfect

[bahmutov]

Instead of loading a static file from the main window, loading localhost website with document.domain=localhost set either in the site itself or in the preload.js script. The first test works

In the above image, the main window opens test window. The test window has opener pointing at the main window. To get all our functions to work, the spec does the following trick

beforeEach(() => {
  if (typeof top !== 'undefined' && top.opener) {
    cy.state('document', top.opener.document)
  }
})

Everything is good - but how do we "clean up" the main window? For example if we reload the main window, the remaining test window is NO LONGER ITS CHILD WINDOW. For example, if I reload the main window, then rerun the tests - they no longer can access opener.document

So maybe the relationship should go the other way: open test window, and let it open the main window to load the Electron application.

[bahmutov]

First we are creating test window that we control - via RDP Before each test it opens child window that loads (via simple http server to have localhost) the main application 'index.html' url - which has node integration enabled, showing process properties in this case.

[bahmutov]

To better understand what potentially might need testing in Electron applications I have randomly picked first apps in https://electronjs.org/apps list The main questions I wanted to see where:

• how was the main window initialized?
• where there multiple windows? (additional windows like config, preferences, about)
• was the main window loaded using loadUrl(...) or via loadFile(<local filename>)?

1. https://github.com/yafp/ttth Chat client

   // Create the browser window.
   mainWindow = new BrowserWindow({
       title: "${productName}",
       frame: true, // false results in a borderless window
       show: false, // hide until: ready-to-show
       width: windowWidth,
       height: windowHeight,
       minWidth: 800,
       minHeight: 600,
       backgroundColor: "#ffffff",
       icon: path.join(__dirname, "app/img/icon/icon.png"),
       webPreferences: {
           nodeIntegration: true,
           webviewTag: true, // # see #37
       }
   });

   There was config window, initialized similarly. Both windows were loaded from file mainWindow.loadFile("app/mainWindow.html")

2. https://github.com/sagargurtu/lector PDF viewer

    win = new BrowserWindow({
        width: 800,
        height: 600,
        minWidth: 300,
        minHeight: 300,
        icon: './src/assets/images/logo.png',
        webPreferences: {
            plugins: true,
            nodeIntegration: true
        },
        frame: false
    });

    // and load the index.html of the app.
    win.loadFile('./src/index.html');

3. https://github.com/CodeF0x/violin music player

function createWindow() {
  window = new BrowserWindow({
    width: 800,
    minWidth: 800,
    height: 600,
    minHeight: 600,
    titleBarStyle: 'hiddenInset',
    useContentSize: false,
    webPreferences: {
      nodeIntegration: true
    }
  });

  window.setResizable(true);
  window.loadFile('src/index.html');
}

4. https://github.com/dot-browser/desktop browser Subclass of BrowserWindow

   export class AppWindow extends BrowserWindow {
   public viewManager: ViewManager = new ViewManager();
   public permissionWindow: PermissionDialog = new PermissionDialog(this);
   public menu: MenuList = new MenuList(this);
   
   constructor() {
   super({
     frame: false,
     minWidth: 500,
     minHeight: 450,
     width: 1280,
     height: 720,
     show: false,
     backgroundColor: '#1c1c1c',
     title: 'Dot Browser',
     titleBarStyle: 'hiddenInset',
     maximizable: false,
     webPreferences: {
       plugins: true,
       nodeIntegration: true,
       contextIsolation: false,
       experimentalFeatures: true,
       enableBlinkFeatures: 'OverlayScrollbars',
       webviewTag: true,
     },
     icon: resolve(app.getAppPath(), '/icon.png'),
   });
   }
   ...

Interestingly loads file:// url in production

if (process.env.ENV === 'dev') {
      this.setIcon(nativeImage.createFromPath(resolve(app.getAppPath() + '\\static\\icon.png')))
      this.webContents.openDevTools({ mode: 'detach' });
      this.loadURL('http://localhost:4444/app.html');
    } else {
      this.loadURL(join('file://', app.getAppPath(), 'build/app.html'));
    }

5. https://github.com/digimezzo/knowte-electron Note-taking application

    mainWindow = new BrowserWindow({
      'x': mainWindowState.x,
      'y': mainWindowState.y,
      'width': mainWindowState.width,
      'height': mainWindowState.height,
      backgroundColor: '#fff',
      frame: false,
      icon: path.join(__dirname, 'build/icon/icon.png'),
      show: false
    });

Again, uses loadUrl to load file:// url

  mainWindow.loadURL(url.format({
        pathname: path.join(__dirname, 'dist/index.html'),
        protocol: 'file:',
        slashes: true
      }));

[bahmutov]

Trying to load main application window via file:// url does not give our localhost test window access to it

[brian-mann]

We need to experiment to see if properly enabling the proxy on the browser window / electron routes requests to the file protocol through the cypress proxy. If it does not then we'll need to go down the plugin route and require users pass us the browser window instance to our plugin like so...

const cypressElectron = require('@cypress/electron-plugin')

const win = new BrowserWindow(...)

cypressElectron(win)

Once we receive the browser window, we could override the loadFile function to properly re-route through the proxy, forcing us to receive it.

Alternatively we could monkey patch the electron API's themselves such as BrowserWindow.prototype.loadURL and re-route that through us.

I'm hoping we can just go down the regular network proxy as long as chromium routes requests to the file:// protocol through us.

[bahmutov]

I am not sure what the re-routing loadFile would do. When we re-route HTML load via our proxy it is to insert document.domain = 'localhost' there. When we re-reroute file load and even if the index.html file has the domain set

<script>
  document.domain = 'localhost'
</script>

it does not work - here is a screenshot

I think we might be able to run OUR window / iframe from domain '' by loading it from a file too

[bahmutov]

Question: can we proxy external domain and load it correctly (by inserting document.domain = 'localhost' into the returned HTML). For example, if the Electron application does

open('http://todomvc.com/examples/vue/', 'mainwindow', replace)

Trying to load, getting cookie error - because we now have 2 windows - the test window and the child Electron window with the app. Our automation works against the test window (where the command log is), but we also need a second connection to drive the application window and set the cookies, everything correctly there.

Hmm, do not see the domain loaded by the open('https://todomvc.com') going through our proxy for some reason.

[bahmutov]

Progress meeting Wed 08-14-2019

Showed demo testing localhost page: loading, accessing Node integration, clicking and typing. The main window loads our test runner, and opens a web window for the main application to load like this

const mw = (window.mw = open(
  'http://localhost:4600',
  'mainWindow',
  replace
))

The test runs as shown in this animation. Note that direct access to the child window allows us to see DOM snapshots, etc

Next steps

We are really interested in writing down little "seams" that Cypress test runner should expose for the Electron testing plugin code to use to change the test runner behavior.

1. how to display + detect we should add electron as a selectable browser?

   • right now it's hard-coded to concat browsers. We probably want to expose in plugins file (future cypress.js file that replaces plugins.js and cypress.json file)

2. @flotwig to look at how file:// protocol is routed through the proxy?

   • look into custom protocol handler? Or interception at the file system layer or loading file via local HTTP server, so it becomes 'localhost'

3. expose a way to tap into how cy.visit() is resolved

   • enables you to change from using $autIframe.contentWindow.location.href to using window.open(…). Current trick I do in the spec file is

     const mw = (window.mw = open(
     'http://localhost:4600',
     'mainWindow',
     replace
     ))
     // TODO resolve when mw.document is valid
     // for now simply wait
     setTimeout(() => {
     cy.state('document', mw.document)
     cy.state('window', mw)
     resolve()
     }, 100)

4. control the app's browser window scaling the same way as the $autIframe

   • possibly not even allow the browserwindow to be detached
   • making it work the same as it currently does
   • possibly replace the <iframe> with the <webview>
     • Electron's BrowserView class is the replacement for <webview>
   • maybe allow the user to control whether its detached or not
   • embedding allows us to capture the video in a single place Approximately this will look like this:

5. normalize to use the RDP instead of launch args for settings things like the network proxy

   • this is immediate work that can be done now
   • this can work the same for both chrome, for electron vanilla browser, or custom 3rd party native electron apps

6. look at Spectron

   • what APIs do they allow or add to Electron app testing?
   • right now we are treating the loaded site in the test window / view as a normal website. But what about testing Node integration, accessing Electron APIs etc from the test?

Next goal

• do not worry too much about a proxy for 3rd party sites, aside from localhost.
• set the proxy on the Electron browser process using RDP command and point back at Cypress process
• switch the child window to be a subview of the main window.
  • work on cy.visit to do all the work for switching / loading the site url
• start work on exposing the list of browsers from detected-only per machine to being per project. Probably needs some logic in plugins.js file or more likely in cypress.js

[bahmutov]

Testing proxy CLI arguments to Electron application

1. When launching Electron with same CLI args as Chrome --proxy-server=http://localhost:5556,--proxy-bypass-list=<-loopback> we get our proxy to see and proxy external sites. For example, if the app window is loading http://todomvc.com

   cypress:server:proxy handling proxied request { url: '/', proxiedUrl: 'http://todomvc.com/', headers: { host: 'todomvc.com', 'proxy-connection': 'keep-alive', 'upgrade-insecure-requests': '1', 'user-agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_12_6) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/76.0.3809.102 Electron/6.0.1 Safari/537.36', accept: 'text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,image/apng,*/*;q=0.8,application/signed-exchange;v=b3', referer: 'http://localhost:5556/__cypress/iframes/integration/spec.js', 'accept-encoding': 'gzip, deflate', 'accept-language': 'en-US', cookie: '_ga=GA1.2.665055498.1565718779' }, remoteState: { auth: undefined, props: null, origin: 'http://localhost:5556', strategy: 'file', visiting: undefined, domainName: 'localhost', fileServer: 'http://localhost:52703' } } +39ms

Without --proxy* CLI arguments the requests from the Electron application do not appear in our proxy.

See Electron issue https://github.com/electron/electron/issues/584 for --proxy-server and the full list of CLI switches https://github.com/electron/electron/blob/master/docs/api/chrome-command-line-switches.md

Conclusion

We can use same proxy CLI arguments with Chrome and Electron external application

[brian-mann]

We could override the loadFile and loadURL methods on the browser window and force those requests via the file:// protocol to be routed through the proxy. There are lots of ways to solve this.

Regardless though - we have to initially suppress the app from initially loading / inflating - and must have it load cypress instead. This is really simply handled by the user modifying their code (or for instance passing off the browser window instance to us).

Then they would use cy.visit(...) directly to load in their application. The only other problem here is that their code could be instantiating other browser windows, which then have the exact same problem. I believe the solution here would be to require / import us as a module in their code (conditionally using a flag) and then we'd be able to monkey patch the BrowserWindow directly. Alternatively... since we have direct access to the initial BrowserView instances, we may be able to do this ourselves automatically.

[flotwig]

I played around some with trying to intercept loadFile and loadURL without overriding the functions.

• You can intercept loadFile using protocol.interceptHttpProtocol (or protocol.interceptStreamProtocol, presumably others) and attempt to redirect the request to the Cypress proxy:

  protocol.interceptStreamProtocol('file', (req, cb) => {
    console.log('file:// request intercepted', req)
  
    // imagine this `rp` is for Cypress's proxy
    rp({
      headers: req.headers,
      method: req.method,
      url: 'https://httpdump.io/vj6ef?originalFileUrl=' + req.url,
      body: req.uploadData ? req.uploadData : null,
      resolveWithFullResponse: true
    }).then((res) => {
      cb({
        data: res.body,
        statusCode: res.statusCode,
        headers: res.headers
      })
    })
  })

  However, this won't work - with webSecurity on, the BrowserWindow just shows , without it, the BrowserWindow silently shows nothing on a loadFile. This could probably only be used to intercept HTTP URLs. There is a protocol.interceptFileProtocol, but it can only be used to serve files from disk in response to a file:// request.

• Network.setRequestInterception and Fetch.enable from RDP don't seem to intercept file:// requests, so they can't be used.

In light of this, overriding loadFile and loadURL in JS sounds like the cleanest way to intercept those calls.

[bahmutov]

I have looked at using https://electronjs.org/docs/api/browser-view It is like child frameless window, except has several downsides

Example code

// main-browser-view.js
// $ npx electron ./main-browser-view.js
const { app, BrowserView, BrowserWindow } = require('electron')

function createWindow () {
  let win = new BrowserWindow({ width: 800, height: 600 })
  win.on('closed', () => {
    win = null
  })
  win.loadFile('main-browser-view.html')

  let view = new BrowserView()
  win.setBrowserView(view)
  view.setBounds({ x: 0, y: 150, width: 800, height: 450 })
  view.webContents.loadURL('https://electronjs.org')
}

app.on('ready', createWindow)

<body>
  <h1>Browser view demo</h1>
  <p>this page embeds browser view instance below</p>
</body>

The loaded view literally sits on top of everything in the top window, even the DevTools.

But the browser view instance is "invisible" to the outside window JavaScript, trying to solve this conundrum.

Hmm, seems I cannot get reference to the window object inside a BrowserView - only by using window.open or its equivalent can I get the reference back that allows accessing child window. Need to investigate webview

[bahmutov]

<webview> has been deprecated / disabled by default https://electronjs.org/docs/api/webview-tag#webview-tag ughh.

Back to square 1 - just use our current iframe method to visit the site. Since the first browser window running in the external Electron the iframe has access to the Node stuff too

[brian-mann]

We could still instantiate a separate browser window - it would just make recording more difficult. We'd need to multiplex in two different video streams and align them so it appears in a single video. Not the hardest thing, but not an instant win either.

[bahmutov]

Side note

I was thinking about the code I used to open the second window from Electron app

beforeEach(function openSeparateWindow () {
  return new Promise(resolve => {
    const replace = true
    const mw = (window.mw = open(
      'http://example.cypress.io',
      'mainWindow',
      replace
    ))

    // TODO resolve when mw.document is valid
    // for now simply wait
    setTimeout(() => {
      cy.state('document', mw.document)
      cy.state('window', mw)
      resolve()
    }, 1000)
  })
})

as I suspected - this works even today in v3.4.1 with external Chrome browser as well. This might be good for people who have problems testing their web apps in an iframe

[brian-mann]

There's no technical difference between opening a window and using it in an iframe. All the same rules apply. Maybe dev tools binds slightly differently, but probably not.

On Thu, Aug 15, 2019 at 4:57 PM Gleb Bahmutov notifications@github.com wrote:

Side note

I was thinking about the code I used to open the second window from Electron app

beforeEach(function openSeparateWindow () { return new Promise(resolve => { const replace = true const mw = (window.mw = open( 'http://example.cypress.io', 'mainWindow', replace ))

// TODO resolve when mw.document is valid
// for now simply wait
setTimeout(() => {
  cy.state('document', mw.document)
  cy.state('window', mw)
  resolve()
}, 1000)

}) })

as I suspected - this works even today in v3.4.1 with external Chrome browser as well. This might be good for people who have problems testing their web apps in an iframe

[image: Screen Shot 2019-08-15 at 4 55 11 PM] https://user-images.githubusercontent.com/2212006/63126339-aaddf180-bf7d-11e9-8eb6-0b914e5f8243.png

— You are receiving this because you commented. Reply to this email directly, view it on GitHub https://github.com/cypress-io/cypress/issues/4964?email_source=notifications&email_token=AAJVZ4HPDMM6MK52FHBJVCDQEW7KFA5CNFSM4IKWSNBKYY3PNVWWK3TUL52HS4DFVREXG43VMVBW63LNMVXHJKTDN5WW2ZLOORPWSZGOD4M7JFQ#issuecomment-521794710, or mute the thread https://github.com/notifications/unsubscribe-auth/AAJVZ4FOXLADANZR5WEMKFLQEW7KFANCNFSM4IKWSNBA .

[bahmutov]

Yup, so I think we can advise this as a workaround to people struggling with their site being iframed

[brian-mann]

We could however provide this as an option if there are ergonomic / UI / UX upsides.

[brian-mann]

Framebusting code will still apply even if its being window.open. There's no real technical difference. window.top !== window.self. It won't offer any upside for those people unfortunately - but it shouldn't matter since we properly handle that by rewriting obstructive code anyway.

[bahmutov]

aren't there people that use width of the top parent when calculating sizes?

[brian-mann]

We rewrite all references to top to point them to self so that its always bounded to the window - so that ultimately it points to the right thing irrespective. We'd have to do that in this case too - we won't get anything for free. This would just be an ergonomic/UI/UX change, wouldn't help or affect anything else.

[brian-mann]

It would also show the URL bar as well, which could be nice for some people. They wouldn't have to look at the URL bar that we render.

[bahmutov]

yeah, it does show the url of the site in the child window

[brian-mann]

I just tried this out - and the other advantage would be that dev tools can be bound only to the AUT - which would only display the DOM + network requests for the AUT. And likely hide cypress requests.

[brian-mann]

But every time the window is closed + reopened it would close devtools. Another option would be to embed the devtools into Cypress (remotely connected to the other window) so that it wouldn't constantly get unbound or closed. That could have some real upside.

[bahmutov]

that's actually a good one for users - not worry about devtools context

[brian-mann]

I wonder if we could just do this even when the window is in an iframe... it would be worth exploring.

[bahmutov]

One other thing - this blows away the window before each test! no more lifecycle leaking callbacks and requests from one test into another

[bahmutov]

BTW, the DevTools just stay open through the tests

[brian-mann]

The devtools stay open because the window isn't being closed. It would close if we closed the window.

Architecturally this is no different than an iframe and it would inherit all of the same problems. This doesn't solve lifecycle at all. The problem with lifecycle isn't our ability to close the window or iframe (we could do that now). The challenges are around the API's and controlling when you want to blow away the window vs recreating it and forcing users to cy.visit() over and over again.

[bahmutov]

Right, but since we haven't implemented lifecycle events, I would advise users with random crashes and lots of XHR requests to do the above window open to take control of the window themselves.

[brian-mann]

Functionally or architecturally it won't change anything. The upsides are purely visual and superficial.

[brian-mann]

But don't get me wrong - I actually think those visual / ergonomic upsides may be worth exploring because we have a lot of options like embedding the devtools into the cypress window and mounting it against the window and/or iframe (i have to try that out). We could then programmatically control the devtools to do things like automatically opening / closing it when you interact with the command log.

I also think at the very least we could offer opening in a separate window as an option to users that they could control themselves. It wouldn't do this in cypress run mode, so the video would not be affected. Functionally nothing has to change in cypress to support this.

[bahmutov]

Exploring the application's side of testing

Aside from loading and controlling the user application from Cypress, we need to think what the entire E2E test is capable of testing:

• is it just the browser page layout?
• is the only reset before each test is the browser page reload in cy.visit() or do the apps need more actions in the Electron source to reset?
• what does Spectron allow?

Spectron examples

Typical test (via Ava)

import test from 'ava';
import {Application} from 'spectron';

test.beforeEach(async t => {
  t.context.app = new Application({
    path: '/Applications/MyApp.app/Contents/MacOS/MyApp'
  });

  await t.context.app.start();
});

test.afterEach.always(async t => {
  await t.context.app.stop();
});

test(async t => {
  const app = t.context.app;
  await app.client.waitUntilWindowLoaded();

  const win = app.browserWindow;
  t.is(await app.client.getWindowCount(), 1);
  t.false(await win.isMinimized());
  t.false(await win.isDevToolsOpened());
  t.true(await win.isVisible());
  t.true(await win.isFocused());

  const {width, height} = await win.getBounds();
  t.true(width > 0);
  t.true(height > 0);
});

[bahmutov]

Ok, I got it. For all additional resetting that the app might want to do before loading the page, it should just use actions similar to cy.task via Electron's native ipcMain and ipcRenderer communication.

Example: Imagine the app needs to reset the global counter that the page wants to show

// main.js
global.counter = 0

<h2>Global counter</h2>
<div id="counter"></div>
<script>
  let remote = require('electron').remote
  document.getElementById('counter').innerHTML = remote.getGlobal('counter')
</script>

Then the main.js would change like this:

// main.js
global.counter = 0

// let's give Cypress tests access to the "global.counter"
const tasks = {
  setCounter (n) {
    console.log('setting global.counter to', n)
    global.counter = n
  }
}
// this is all generic code that WE can hide in `@cypress/electron-utils`
ipcMain.on('task', (e, taskName, ...args) => {
  console.log('ipMain task "%s"', taskName)
  if (tasks[taskName]) {
    // TODO handle sync and promise-returning tasks
    const result = tasks[taskName](...args)
    e.returnValue = result
  } else {
    console.error('Unknown task name "%s"', taskName)
    e.returnValue = null
  }
})

and the spec code

// spec.js
Cypress.Commands.add('electronTask', (name, ...args) => {
  // avoid Cypress bundling using its own "require"
  const ipcRenderer = global['require']('electron').ipcRenderer
  return ipcRenderer.sendSync('task', name, ...args)
})
it('resets app counter', () => {
  cy.electronTask('setCounter', 5)
  cy.visit('http://localhost:4600')
  cy.get('#counter').should('have.text', '5')

  cy.electronTask('setCounter', 21)
  cy.reload()
  cy.get('#counter').should('have.text', '21')
})

[bahmutov]

Hmm, if we allowed system Node that executes cypress/plugins/index.js file to BE the local Electron app, then the plugins could BE the main.js or our wrapper - becoming background.js file like we want in Cypress v5

Then the app's own main.js would initialize the application, pop main window pointing out our test runner url AND each cy.task would directly have access to all app's variables one might want to control during testing. This could be sweet

[bahmutov]

Future work estimate

We probably can limit the initial work to 3 parts

• a programmatic way to load local Electron application and local main.js script file - a week of work, depending on how much we do for cypress.json to cypress.js transition. This will require at least some Cypress Test Runner changes + documentation
  • investigate if today we can do this using plugin code to manipulate the list of browsers in the config + validate the returned information
  • handle external electron application similar to Chrome browser - control cookies and the rest via Chrome Remote Interface
• some docs explaining how to use CLI to the electron application to either do "normal" load or let Cypress load in the main window - a day or two
  • we launch electron main.js
  • the app opens browser window - and Cypress can control it because we pass remote interface CLI argument to their Electron
  • the app should show Cypress GUI - maybe via CLI argument
• a utility for adding cy.task equivalent for Electron applications to work with the main process from the spec file - a week of work, this is part of external plugin + documentation
• document how users can "pop" the window we spawn instead of using cy.visit inside an app iframe

Limitations

• probably loading an url at first, I have not checked if Cypress built-in server can serve the local src/index.html as well as Electron can do window.loadFile('src/index.html')

[FrancescoBorzi]

This link is not working: https://github.com/cypress-io/cypress-example-electron

[bahmutov]

@FrancescoBorzi I have just made this repo public, but it not ready, and is really in a state of experimentation

[bahmutov]

Reworked architecture using a private sandbox project:

• supports the browser window created by the application, not just reusing Cypress window. You have your application window created on-demand via a window factory function.
  • we believe most Electron applications (like the ones on https://electronjs.org/apps) need to create their browser window in order to function properly, and just overwriting cy.visit to load an url is not going to be enough
• normal workflow
• Cypress testing workflow
• steps to implement
  • load a local Electron per-project
  • allow users to control list of browsers https://github.com/cypress-io/cypress/pull/5068
  • allow users to return an external Electron app in the list of browser and launch it similarly to new Chrome (and control via Chrome remote interface, uses https://github.com/cypress-io/cypress/pull/4628)
  • run tests similar to the sandbox demo, polish the experience, release as a plugin for faster iteration

We are still thinking about how to cleanly separate loading an application and creating its window to be tested.

[bahmutov]

Updated work estimate

Roughly 3 parts:

1. use external Electron as a browser
2. open and control user app's window
3. ergonomics and additional features

Part 1: use external Electron as a browser: 1 week

• a programmatic way to load local Electron application and a user-specified local main.js script file - a week of work.
  • there is already PR to let the user control list of browsers from the plugins file https://github.com/cypress-io/cypress/pull/5068
  • allow returning external Electron path and maybe some way of saying "load Cypress Electron plugin to test it". Maybe even path to the factory function that creates the main browser window?
  • Cypress Test Runner changes + documentation
• run external Electron application when using cypress run command (right now hard-coded to run the built-in Electron browser)

Part 2: open and control user app's window: 1 week

• handle the external electron application similar to Chrome browser - control cookies, record video and the rest via Chrome Remote Interface via https://github.com/cypress-io/cypress/pull/4628
• some docs explaining how to use CLI to the electron application to either do "normal" load or let Cypress load in the main window, similar to electron-sandbox demo
• publish the Electron custom command as a separate plugin @cypress/electron-plugin

Part 3: ergonomics and examples: 1 to 2 weeks

• Need to polish the ergonomics of testing Electron apps. Everything from app reload, to event listeners, to IPC equivalent of cy.task, etc.
• list of apps to try testing at https://electronjs.org/apps
• take care of unexpected bugs found

Plan

• [x] build Cypress binary with Electron support from a branch (at least Mac binary) on every commit, keep merging separate branches and develop branch into this branch as needed. This allows anyone to install a new version of Cypress using environment variable CYPRESS_INSTALL_BINARY=<url> npm i <url>
• [ ] document and set access to be public for @cypress/electron-plugin package
• [ ] think if we can simplify installation of beta versions of Cypress https://github.com/cypress-io/cypress/issues/5135

[bahmutov]

Used browser list PR https://github.com/cypress-io/cypress/pull/5068 to insert the custom Electron browser from plugins/index.js file in the project

module.exports = (on, config) => {
  // `on` is used to hook into various events Cypress emits
  // `config` is the resolved Cypress config

  // remove "standard" browsers and use
  // our local Electron as a browser
  config.browsers = [
    {
      name: 'electron-sandbox',
      family: 'electron-app',
      displayName: 'electron-sandbox',
      version: pkg.version,
      // return full path to Electron module
      path: path.join(
        __dirname,
        '..',
        '..',
        'node_modules',
        '.bin',
        'electron'
      ),
      // show full package version in the browser dropdown
      majorVersion: `v${pkg.version}`,
      info:
        pkg.description || 'Electron.js app that supports the Cypress launcher'
    }
  ]

  return config
}

nice, only a single browser

Probably need to pass additional CLI arguments to Electron, like our main file path

[kettanaito]

Hello. Thank you for the great work behind this feature!

If I may, would such testing be automating an actual Electron app, or the application it's serving? I'm wondering what would be the testing environment: is it native Electron, or Cypress replacing electron and server+automating the application?