ECONNREFUSED error when stopping multiple spectron applications

[alxroyer]

Short description

When several Electron applications have been launched with spectron, it seems that calls to .stop() fail as soon as the first application has been stopped.

Configuration

• OS: Windows 10
• node v10.15.1
• electron v4.1.3
• spectron v5.0.0

Related issues

Root cause could be similar to #186, or #272? To be checked.

---

Investigations / steps to reproduce

Assume we have an Electron application ready, for instance the electron-quick-start app:

git clone https://github.com/electron/electron-quick-start
cd electron-quick-start/
npm install
npm install --save-dev spectron

app.js

Add the following 'app.js' file in the 'electron-quick-start' directory:

const spectron = require("spectron");
const path = require("path");

const electronPath = path.join(__dirname, 'node_modules', '.bin', 'electron') + (process.platform === "win32" ? ".cmd" : "");
const appPath = __dirname;

const apps = {};

exports.startApp = function(appName) {
    return new Promise(async function(resolve, reject) {
        try {
            const app = apps[appName] = new spectron.Application({
                path: electronPath,
                args: [appPath]
            });
            await app.start();
            await app.client.waitUntilWindowLoaded();
            console.log(`Application '${appName}' successfully started`);
            resolve();
        } catch (err) {
            console.error(`Error while starting application '${appName}': ${err}`);
            reject(err);
        }
    });
}

exports.stopApp = function(appName) {
    return new Promise(async function(resolve, reject) {
        try {
            const app = apps[appName];
            await app.stop();
            console.log(`Application '${appName}' successfully stopped`);
            resolve();
        } catch (err) {
            console.error(`Error while stopping application '${appName}': ${err}`);
            reject(err);
        }
    });
}

fifoFails.js

Let's start several Electron applications with spectron, then stop them. Add the following 'fifoFails.js' file in the 'electron-quick-start' directory:

const { startApp, stopApp } = require("./app");

async function main() {
    try {
        // Start apps.
        await startApp("A");
        await startApp("B");
        // Stop apps.
        await stopApp("A");
        await stopApp("B");
        // Success.
        console.log("Success");
    } catch (err) {
        // Failure.
        console.error("Error: " + err.toString());
    }
}
main();

Then execute:

$ node fifoFails.js
Application 'A' successfully started
Application 'B' successfully started
Application 'A' successfull stopped
Error while stopping application 'B': Error: connect ECONNREFUSED 127.0.0.1:9515
Error: Error: connect ECONNREFUSED 127.0.0.1:9515

Stopping the application 'B' ends with a ECONNREFUSED error.

lifoWorks.js

If FIFO fails, let's test LIFO. Add the following 'lifoWorks.js' file in the 'electron-quick-start' directory:

const { startApp, stopApp } = require("./app");

async function main() {
    try {
        // Start apps.
        await startApp("A");
        await startApp("B");
        await startApp("C");
        // Stop apps.
        await stopApp("C");
        await stopApp("B");
        await stopApp("A");
        // Success.
        console.log("Success");
    } catch (err) {
        // Failure.
        console.error("Error: " + err.toString());
    }
}
main();

The test ends with success:

$ node lifoWorks.js
Application 'A' successfully started
Application 'B' successfully started
Application 'C' successfully started
Application 'C' successfull stopped
Application 'B' successfull stopped
Application 'A' successfull stopped
Success

mixWorking.js

Sounds wierd it could simply be a matter of FIFO / LIFO. Let's test a little bit further what works or not. Add the following 'mixWorking.js' file in the 'electron-quick-start' directory:

const { startApp, stopApp } = require("./app");

async function main() {
    try {
        // Start apps.
        await startApp("A");
        await startApp("B");
        await startApp("C");
        // Stop apps.
        await stopApp("B");
        await stopApp("C");
        await stopApp("A");
        // Success.
        console.log("Success");
    } catch (err) {
        // Failure.
        console.error("Error: " + err.toString());
    }
}
main();

The test still ends with success, even though we have swtched applications B and C:

$ node mixWorking.js
Application 'A' successfully started
Application 'B' successfully started
Application 'C' successfully started
Application 'B' successfull stopped
Application 'C' successfull stopped
Application 'A' successfull stopped
Success

mixFailing.js

So what's the configuration that makes it fail? Add the following 'mixFailing.js' file in the 'electron-quick-start' directory:

const { startApp, stopApp } = require("./app");

async function main() {
    try {
        // Start apps.
        await startApp("A");
        await startApp("B");
        await startApp("C");
        // Stop apps.
        await stopApp("B");
        await stopApp("A");
        await stopApp("C");
        // Success.
        console.log("Success");
    } catch (err) {
        // Failure.
        console.error("Error: " + err.toString());
    }
}
main();

B can be stopped successfully, but C (which is now behind A) fails stopping:

$ node mixFailing.js
Application 'A' successfully started
Application 'B' successfully started
Application 'C' successfully started
Application 'B' successfull stopped
Application 'A' successfull stopped
Error while stopping application 'C': Error: connect ECONNREFUSED 127.0.0.1:9515
Error: Error: connect ECONNREFUSED 127.0.0.1:9515

Conclusion

It seems that stopping the first application launched kills spectron resources, which prevents other applications to be stopped from that point, unless anything else is wrong in my code & configuration... :-) Let me know !

[MarkGeeRomano]

My experience exactly. I put a setTimeout in my afterEach hook for 10s to prevent this (using AVA).

[alxroyer]

10s... Geez! Makes me feel that the spectron technology is not ready for automated system testing yet. :-(

[senyszrm]

So I ran into this same issue, and what I found out after debugging was that with running multiple applications, they seemed to be having app.client connect to 127.0.0.1:9515. It seems like when the first Spectron app stops, it calls app.client.end and that may shut down that shared client for ALL apps, which stops the rest of them from doing anything with their own version of app.client (including stopping theirs). The simple fix in my case was to add in the following where we had an array pool of multiple applications called appPool:

        let app = new Application({
            ... // other parameters here
            port: 9515 + appPool.length, // This is the important line
        })
        appPool.push(app)

I noticed that in your example code you weren't setting different ports per Spectron Application. Maybe it's the same issue?

[alxroyer]

Interesting info! A quick and safe workaround indeed. Neverthless, I still believe something could be fixed in spectron. Thanks for sharing @senyszrm.