cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.01k stars 3.18k forks source link

After test reloads due to saving spec file, cy.contains starts asserting existence when it shouldn't #27403

Closed FFdhorkin closed 1 year ago

FFdhorkin commented 1 year ago

Current behavior

This seems to only happen when chai-parentheses or dirty-chai is installed, so this may very well be a bug in those packages, but it could also be an issue with how Cypress uses chai behind the scenes, so I'm posting here. I'm notifying the chai-parentheses dev as well.

I'm working on a modified version of the cypress-kitchen-sink's todo app tests.

This line is where I discovered the issue: cy.contains('Pay electric bill').should('not.exist');

If you run on a clean instance of Cypress, the test passes as expected. If chai-parentheses is installed, it appears the final assertion doesn't get executed, but the test does pass on the first run.

If you are in the spec file in vscode and hit save, when Cypress reruns the test suite automatically, you get a failure. You can also trigger this behavior by clicking the "Run all tests" ♻️ button in the UI.

It appears that on the reloading of the code, cy.contains('Pay electric bill') asserts the presence of the Pay electric bill, and so it fails before it even gets to the .should('not.exist') (In fact, the exact same result happens if you completely omit the .should('not.exist'))

This does sometimes happen even on a clean run, but it seems to be rarer in that case.

Fresh run (without chai-parentheses): image

Fresh run (with chai-parentheses. Note line 8 is missing): image

After hitting save in the test file: image

Desired behavior

Rerunning a test suite should not change the way Cypress commands behave. cy.contains() does not usually assert existence, so on a re-run, it shouldn't, either.

Test code to reproduce

Prerequisite:

First run npm i -D chai-parentheses

Then add this to cypress/support/e2e.ts:

import chaiParentheses from 'chai-parentheses';
chai.use(chaiParentheses);

Reproducable example

describe('example to-do app', () => {
    beforeEach(() => {
        cy.visit('http://localhost:8080/todo')
    });

    context('with a checked task', () => {
        beforeEach(() => {
            cy.contains('Pay electric bill')
                .parent()
                .find('input[type=checkbox]')
                .check();
        });

        it('can filter for uncompleted tasks', () => {
            cy.contains('Active').click();

            cy.get('.todo-list li')
                .should('have.length', 1)
                .first()
                .should('have.text', 'Walk the dog');

            cy.contains('Pay electric bill').should('not.exist');
        });
    });
});

Cypress Version

12.17.2

Node version

18.16.1 (but cypress version shows 16.16.0 bundled)

Operating System

Ubuntu 22.04.2 LTS (via WSL2)

Debug Logs

Github wouldn't allow a log this long; uploaded here: https://appdevtools.com/pastebin/Og5OSd

Other

No response

nagash77 commented 1 year ago

Hi @FFdhorkin , this is not a library that we use or test internally and so I am not surprised to hear it doesn't work. We do a lot of patching to assertion libraries so it's possible it doesn't work with those. We do not have plans at this time to add support for this library explicitly so I am going to close this issue. If you would like to submit a feature request officially asking for support please open a feature request. I do want to set expectations though that this feature will likely be in the backlog until there is more demand for its support.