search

cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
47.1k stars 3.19k forks source link

Cypress not detecting changes which are faster than 50ms #30599

Open michaelbe812 opened 2 weeks ago

michaelbe812 commented 2 weeks ago

Current behavior

I have a application which shows a loading spinner while a request is running.

I have assertions in my cypress tests that the spinner is visible and not visible afterwards.

Sometimes these requests are quite fast (resolving faster than 40-50 ms) and then my tests fail because Cypress does not find the spinner.

Desired behavior

The test should not fail and pass even if the respective ui element is shown only for a very short time

Test code to reproduce

I made here a reproduction: https://github.com/mikelgo/cypress-flaky-test-repro All necessary instructions to reproduce the issue should be included in the repo itself. if more information are needed, let me know :) please note instead of the spinner I just render some text depending on the progress.

In This reproduction I'm just simulating a async task with a setTimeOut. After some trial and error I found out a threshold of 50ms. If a certain piece of UI is visible less than 50ms Cypress is reproducable not able to detect the element. If the element is visible for more than 50ms Cypress is able to detect the element.

Cypress Version

13.13.0

Node version

20.12.2

Operating System

macOS 14.5 (23F79)

Debug Logs

AssertionError
Timed out retrying after 4000ms: expected '<p>' to contain 'Saving...'

Other

No response

jennifer-shehane commented 1 week ago

@mikelgo You can pass a delay option to cy.intercept to mimic a slow response. I would likely have a single test that delays the response and verifies that the loading spinner displays correctly, then have the other tests not verify the loading spinner and just wait for the content to load, so then they're independent of the time the response takes to load.

michaelbe812 commented 1 week ago

thank you @jennifer-shehane for your suggestion. This I can of course do.

In order to understand it: It is just a technical limitation of Cypress/ maybe even browsers in general to detect tasks which are very fast?

jennifer-shehane commented 1 week ago

@michaelbe812 It may be something specific to our implementation or the browser's implementation. Ideally you want to set up a test suite that has predictable, non-variable behavior either way.