kuceb / cypress-plugin-retries

A Cypress plugin to retry failed tests.
MIT License
237 stars 20 forks source link

cypress-plugin-retries causing problems with hanging assertions #64

Closed zbigniewkalinowski closed 4 years ago

zbigniewkalinowski commented 4 years ago

Current behavior:

Tests running from console are hanging on failed assertions while using cypress-plugin-retries As you can see on printscreen this test lasting 645.30s, but it's hanging to eternity

crash

Desired behavior: Test should retry after crash

Versions: Cypress: 4.9.0 cypress-plugin-retries: 1.5.2 Chrome: 83.0.4103.116

ryanvanbelkum commented 4 years ago

I am seeing this issue as well. When running tests on Cypress 4.x, we often see tests hang for hours.

lukeapage commented 4 years ago

I think this started happening around cypress 4.4.1 (don't know exact version)

lukeapage commented 4 years ago

It always happens on the 3rd retry.

Hopefully this feature will hit master soon - but from my investigations of this - If we can disable test:before:run:async event happening during retries it will fix the hangs.

I'l have a go at putting a hack together.

lukeapage commented 4 years ago

hack:

const oldAction = Cypress.action;
Cypress.action = function (event, ...args) {
    if (event === 'runner:test:before:run:async') {
        const runnable = args[1];
        const test = runnable?.ctx?.currentTest || runnable;

        if (test && test._currentRetry > 0) {
            return Cypress.Promise.resolve();
        }
    }
    return oldAction.apply(Cypress, [event, ...args]);
};

I have no idea what the effect of this is, other than it solves the hang issue.

Interestingly after the 2nd retry a test gets very, very slow - something bad is happening, look forward to this functionality reaching core.

nanivijay commented 4 years ago

@lukeapage what is the right place to add this hack? plugins/index.js?

lukeapage commented 4 years ago

Support/index.js but actually it’s caused some of our screenshots to be half written and invalid :(

nanivijay commented 4 years ago

Oh. Then i will wait for a proper fix. Thanks @lukeapage

Cubo25 commented 4 years ago

Having exactly same issue ... hanging assertions. Hopefully somebody will fix this soon ! I meanwhile I will try the hack from @lukeapage so thanks for that. ✋

Cubo25 commented 4 years ago

@zbigniewkalinowski our team find out that if you use for example .should('contains', xxx) then it hangs. We are using just plain cy.contains(xxx) instead and its working fine.

kuceb commented 4 years ago

Retries is now in core in cypress 5.0 so I'm closing this. Please open an issue in the main cypress repo if you're experiencing this in 5.0. Thanks!

zbigniewkalinowski commented 4 years ago

Thanks :) For your information it's reported here: https://github.com/cypress-io/cypress/issues/8363#issuecomment-678938922