cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.86k stars 3.17k forks source link

wrap().invoke().should() ignores timeout options in should assertion #1221

Open peterklijn opened 6 years ago

peterklijn commented 6 years ago

When a function wrapped inside a wrap gets invoked by a invoke, the should seems to ignore the timeout passed along in the wrap object.

Test code:

const slow = () => new Cypress.Promise(resolve => {
  setTimeout(() => resolve('done'), 2000);
});

cy
  .wrap({ slow }, { timeout: 100 })
  .invoke('slow')
  .should('eq', 'done');

This test is a silly example, in reality I'm querying the Mandrill API, which I want to give a very big timeout as Mandrill can take quite a while to show an email, but I do not want to update the defaultCommandTimeout to minutes as other assertions will never take this long.

Additional Info (images, stack traces, etc)

screen shot 2018-01-24 at 11 41 35

screen shot 2018-01-24 at 11 41 51

peterklijn commented 6 years ago

Maybe this is a better example:

const slow = () => new Cypress.Promise(resolve => {
  console.log("I'm being called..")
  setTimeout(() => resolve('done'), 1000);
});

cy
  .wrap({ slow }, { timeout: 100 })
  .invoke('slow')
  .should('eq', 'not done');

screen shot 2018-01-24 at 11 48 03

screen shot 2018-01-24 at 11 48 11

My defaultCommandTimeout is set to 10 seconds.

Here I'd expect the should to stop calling the slow function after one try as the timeout is set to 100ms and the slow function takes 1000ms to respond, however as you can see in the console output it's being called for roughly 10 seconds (you can't see the timestamp, but there's an second interval in each console.log()).

jennifer-shehane commented 6 years ago

Code for .wrap() method can be found here: https://github.com/cypress-io/cypress/blob/develop/packages/driver/src/cy/commands/misc.coffee#L28

ghost commented 6 years ago

Any updates or workarounds for this issue?

peterklijn commented 6 years ago

@jennifer-shehane is there an update on this issue? It is still present in version 3.1.0.

@mitchkm you can do something like this, it's horrible but it works:

let defaultTimeout;

before(() => {
  // Store default timeout
  defaultTimeout = Cypress.config('defaultCommandTimeout');
});

beforeEach(() => {
  // Set the timeout to something high
  Cypress.config('defaultCommandTimeout', 5 * 60 * 1000); // 5 minutes
});

it('long test', () => {
  const slow = () => new Cypress.Promise(resolve => {
    setTimeout(() => resolve('done'), 2 * 60 * 1000);
  });

  cy
    .wrap({ slow })
    .invoke('slow')
    .should('eq', 'done')
    // Change the timeout back to the default so it doesn't effect other tests.
    .then(() => Cypress.config('defaultCommandTimeout', defaultTimeout));
});
jennifer-shehane commented 5 years ago

No work has been done on this issue to date.

gabbersepp commented 4 years ago

Is it intended to pass the timeout down to the should? The documentation states:

You can modify a command’s timeout. This timeout affects both its default assertions (if any) and any specific assertions you’ve added. (https://docs.cypress.io/guides/core-concepts/introduction-to-cypress.html#Applying-Timeouts)

So if you pass a timeout to wrap, this affects wrap and any directly following assertion. e.g. this works:

    it("test", () => {
        cy.visit("https://google.com")
            .wrap({ asd: 200 }, { timeout: 200 })
            .should(e => {
                expect(e.asd).to.eq(1000)
            })
    })

As you have a invoke between the wrap and the should, the timeout will have no effect.

@jennifer-shehane can you please check if I am wrong?

donleqt commented 4 years ago

Here is the solution and example:


/**
 *  helper function that forces cy to wait for a promise
 *
 * @export
 * @param {Promise} promise promise to wait
 * @param {number} [interval=1000] recheck in minlisecons
 * @returns
 */
export function waitPromise(promise, interval = 1000) {
  let isDone = false;

  const runPromise = () => {
    if (isDone) {
      // Wrap and returns the result
      return cy.wrap(promise.catch(error => assert.isNotOk(true, error)));
    }
    return cy.wait(interval).then(() => runPromise());
  };

  // Marks as resolved
  promise.then(() => (isDone = true)).catch(() => (isDone = true));

  return runPromise();
}

And in-use example

it('Restore Password - Should received an email', () => {
  const task = emailHelper.searchLastEmail({
    emailAddress: user.email,
    subject: 'Reset Password',
    timeout: 2 * 60000 // 2 min
  });

  testHelpers.waitPromise(task).then(mail => {
    // Save for later test
    restorePasswordMail = mail;
    return expect(mail).not.to.be.null;
  });
});