Mocha reporter exits before async tasks complete

[ajeetd]

Current behavior:

A custom Mocha reporter that performs asynchronous tasks on receiving the 'end' event cannot complete because the Mocha process is exited. Synchronous actions, however long they may be, can finish.

Mocha has a --exit and --no-exit arg described here: https://mochajs.org/#-exit. By default in Mocha 4.0 + the no-exit arg is supplied which does wait for asychronous functions to complete and so when running in 'vanilla' Mocha this is not an issue, however it looks like Cypress is passing the --exit arg to Mocha.

Desired behavior:

Either change behavior of Cypress to default to Mocha --no-exit or provide a way to pass through args to Mocha so that the --no-exit arg can be passed through to Mocha on cypress run.

Test code to reproduce

Here is an example custom mocha reporter to reproduce the issue. The function called on the 'end' event takes 10 seconds to complete.

// example-reporter.js
const mocha = require('mocha');
module.exports = exampleReporter;

function exampleReporter(runner, options) {
  mocha.reporters.Base.call(this, runner);

  function wait(ms) {
    return new Promise(resolve => setTimeout(resolve, ms));
  }

  runner.on('end', function() {
      console.log('started');
      wait(10000).then(() => console.log('finished'));
  });
}

If you run this using 'vanilla' Mocha where the default is for the --no-exit option is passed you will find the you see both the 'started' and 'finished' logged to the console, 10 seconds apart.

However if you run using Cypress with this command (using the example project):

$(npm bin)/cypress run --spec "cypress/integration/examples/actions.spec.js" --reporter example-reporter.js

You will see that only 'started' is printed to console and Mocha exits. I am specifying a single spec here for clarity, if you run multiple specs you will may see the 'finished' appear when the next spec is running because Mocha is still running, but the point is that, for a custom reporter, it was what happens at the completion of all specs that really matters.

Versions

Tested with Cypress 4.4.1

[cvietor]

I can confirm exactly that behaviour. In our case, we are using the salty-cypress-testrail-reporter, which internally uses axios to make an HTTP Post.

When the last test was run, cypress does not wait for not resolved promises.

Does anyone know any workaround for now?

[ajeetd]

I can confirm exactly that behaviour. In our case, we are using the salty-cypress-testrail-reporter, which internally uses axios to make an HTTP Post.

When the last test was run, cypress does not wait for not resolved promises.

Does anyone know any workaround for now?

I'm lead dev at Tesults and I could not find a workaround using our existing Mocha reporter which is why I opened this issue. However since this was an issue for our customers I need some solution so we ended up creating a new Cypress specific reporter using their Module API. Our users can now use this new Cypress specific reporter to report results to Tesults without issues, the only limitation is that they must start Cypress programmatically due to the use of the Module API as our docs demonstrate, but this probably isn't a major issue if you're running via CI or your automated test system anyway. I don't know if TestRail has as similar solution or you could build one yourself.

[zeburek]

@jennifer-shehane Are there any plans to fix it?

[zeburek]

@cvietor @ajeetd If you're interested - you could use deasync to overcome this issue. I successfully used it in our reporter.

[jprealini]

@zeburek I stumbled upon this issue using the salty-cypress-testrail-reporter Could you provide a little example of how to implement deasync with that reporter? Thanks

[zeburek]

@jprealini https://github.com/qase-tms/qase-cypress/blob/master/src/index.ts#L114

[FFdhorkin]

@jprealini Check out this PR: https://github.com/Vivify-Ideas/cypress-testrail-reporter/pull/16. I don't know if it will get approved, but given salty-cypress-testrail-reporter forks from this repo, you can probably make the same type of changes in salty-cypress-testrrail-reporter

[Amrrx]

Hello, I believe there is a better alternative workaround than using the deasync library.

The Idea

Using the (child_process) Nodejs builtin module, you can start a new sub process from the parent process, in which the main mocha runner is running inside, then you can de attach this child process to run separately in the background even if the parent process is going to die.

The Implementation

Using the (spawn) function from Nodejs, I have successfully solved the issue using such configuration.

spawn('node', [`${__dirname}/distributer.js`], {
                detached: true, // To detach the child process from the parent process
                stdio: 'inherit', // To inherit the parent process stdio which makes the sub process write to the main parent terminal window
                env: Object.assign(process.env, {
                    special_data_config: JSON.stringify(userconfig),
                    test_run_data: JSON.stringify(this.dataObject)
                }) //Some extra data for my reporter
            }).unref();

The (spawn) function is builtin function which takes 3 attributes. 1- A command to be executed (I will run the main (node) command to execute my published file) 2- An array of arguments to be concatenated with the command (I included my publisher file path) 3- An options object which is optional in case you want to pass some extra data and some other configuration.

By default, the parent will wait for the detached child to exit. To prevent the parent from waiting for a given sub process to exit, use the subprocess.unref() method. Doing so will cause the parent's event loop to not include the child in its reference count, allowing the parent to exit independently of the child

By inheriting the stdio, the sub process can write to the main terminal and by default the CI/CD runner cannot ignore.

Doing though, only cause me a little effort to separate my publisher file from the main mocha runner (index) file as they were coupled to each others. then I passed my runner final data to the reporter publisher to the rest on another process.

Keep in mind that you have to pass any extra data as as (String) only and then you can parse and sanitize it inside your child process. And that may be better instead of making your child process read from a file on disk or pass it as a command args which lead to some security issues.

I have implemented such a workaround and it works well on my reporter even on my CI/CD pipeline and I hope it can work with you too.

Nodejs child_process module docs

[rekapallym]

Hello, I believe there is a better alternative workaround than using the deasync library.

The Idea

Using the (child_process) Nodejs builtin module, you can start a new sub process from the parent process, in which the main mocha runner is running inside, then you can de attach this child process to run separately in the background even if the parent process is going to die.

The Implementation

Using the (spawn) function from Nodejs, I have successfully solved the issue using such configuration.

spawn('node', [`${__dirname}/distributer.js`], {
                detached: true, // To detach the child process from the parent process
                stdio: 'inherit', // To inherit the parent process stdio which makes the sub process write to the main parent terminal window
                env: Object.assign(process.env, {
                    special_data_config: JSON.stringify(userconfig),
                    test_run_data: JSON.stringify(this.dataObject)
                }) //Some extra data for my reporter
            }).unref();

The (spawn) function is builtin function which takes 3 attributes. 1- A command to be executed (I will run the main (node) command to execute my published file) 2- An array of arguments to be concatenated with the command (I included my publisher file path) 3- An options object which is optional in case you want to pass some extra data and some other configuration.

By default, the parent will wait for the detached child to exit. To prevent the parent from waiting for a given sub process to exit, use the subprocess.unref() method. Doing so will cause the parent's event loop to not include the child in its reference count, allowing the parent to exit independently of the child

By inheriting the stdio, the sub process can write to the main terminal and by default the CI/CD runner cannot ignore.

Doing though, only cause me a little effort to separate my publisher file from the main mocha runner (index) file as they were coupled to each others. then I passed my runner final data to the reporter publisher to the rest on another process.

Keep in mind that you have to pass any extra data as as (String) only and then you can parse and sanitize it inside your child process. And that may be better instead of making your child process read from a file on disk or pass it as a command args which lead to some security issues.

I have implemented such a workaround and it works well on my reporter even on my CI/CD pipeline and I hope it can work with you too.

Nodejs child_process module docs

can you please provide an example?

[kmieshkov]

The solution with spawn is more elegant, but it didn't work for me as it is presented here and didn't work in my Bitbucket Pipeline as well. But with spawnSync it works perfectly, even more, currently logs from the distributer file (in my case importXrayReport.js) print in the same stream with the current reporter and don't mix up with the next ones

        spawnSync('node', [`${__dirname}/importXrayReport.js`], {
          detached: true,
          stdio: 'inherit'
        });

[gustawx]

Hello Cypress team, any update on this issue?

[etaylork]

Hello Cypress team, do we have any update or resolution to work around this issue?