codeceptjs / CodeceptJS

Supercharged End 2 End Testing Framework for NodeJS
http://codecept.io
MIT License
4.11k stars 724 forks source link

Errors in async function of PageObjects are catched in Before #1109

Closed rpetersen27 closed 6 years ago

rpetersen27 commented 6 years ago

I tried to setup async stuff with a page object in the Before function of a Testsuite. If the asynchronous code fails, the error is catched and the test proceeds. I broke down the issue to a minimal example (which is still quite big):

// codecept.conf.js
module.exports = {
  ...
  'include': {
        'test': './test'
   },
  ...
};
// scenario.js
Feature('Test');

Before(async function (test) {
    await test.foo();
});

Scenario('Test', function () {
});
// ./test.js
module.exports = {
  foo: async function () {
    await bar();
  },
};

Details

As far as I can see, the error is catched by the asyncWrapper (in codeceptjs/lib/container.js) and saved inside the recorder as 'asyncError' but is not thrown after the end of Before. I would expect the test to fail in Before because of that async error.

xeonarno commented 6 years ago
Before(async function (test) {
    await test.foo();
});

I am maybe wrong but I think you must add a Promise between nested await :

Before(async function (test) {
    await Promise.resolve(test.foo());
});

Concerning your error, isn't it better to catch the error directly

Before(async function (test) {
try{
    await test.foo();
}catch(e){
  console.log(e);
}
});

hope it helps !

rpetersen27 commented 6 years ago

No, nesting await does require an additional Promise. Async creates a promise on the fly.

I already tried to to use a try-catch in the Before-function. But the error is not thrown in the Before-function because the foo function is part of a PageObject and all functions of page-objects are wrapped inside a function which catches all errors. I guess this behavior is OK to prevent that the execution stops at the point of the error. But the error should be considered as soon as the Before-function is over. The same is done in the Scenario itself. If you call test.foo() inside the Scenario, you would not be able to catch the error there, but after the Scenario is executed, Codecept will display that error message.

johnyb commented 6 years ago

I also played around with this a little longer. If you add a failing async actor function, like await I.grabValueFrom('#foo'); just after await test.foo(), the scenario above will fail with the correct (first thrown) error.

So the problem is indeed that errors in page objects are silently ignored and can not fail a Before. Actors seem to work differently and actually do fail a Before. The logic in codeceptjs/lib/container.js does work in the sense, that it stores a reference to the very first error being thrown.

johnyb commented 6 years ago

In case anybody is also running into this, a workaround would be to avoid using async functions in the support object:

// ./test.js
module.exports = {
  foo: function () {
    return new Promise(function (resolve) {
        bar();
        resolve();
    });
  },
};