cypress-io / cypress-realworld-app

A payment application to demonstrate real-world usage of Cypress testing methods, patterns, and workflows.
https://docs.cypress.io
MIT License
5.44k stars 2.18k forks source link

Flaky E2E ui/notifications.spec.ts test #1332

Closed MikeMcC399 closed 1 year ago

MikeMcC399 commented 1 year ago

On

Execute:

yarn dev

and in separate terminal execute:

yarn cypress:run

The following test sometimes fails and sometimes succeeds:

cypress/tests/ui/notifications.spec.ts

For example, in run 8376810322


FAILING RUN

  Running:  ui/notifications.spec.ts                                                      (14 of 21)

  Notifications
    ✓ renders an empty notifications state (5284ms)
    notifications from user interactions
      (Attempt 1 of 3) User A likes a transaction of User B; User B gets notification that User A liked transaction 
      (Attempt 2 of 3) User A likes a transaction of User B; User B gets notification that User A liked transaction 
      1) User A likes a transaction of User B; User B gets notification that User A liked transaction 
      ✓ User C likes a transaction between User A and User B; User A and User B get notifications that User C liked transaction (46425ms)
      ✓ User A comments on a transaction of User B; User B gets notification that User A commented on their transaction (4497ms)
      ✓ User C comments on a transaction between User A and User B; User A and B get notifications that User C commented on their transaction (6625ms)
      ✓ User A sends a payment to User B (3857ms)
      ✓ User A sends a payment request to User C (3073ms)

  6 passing (2m)
  1 failing

  1) Notifications
       notifications from user interactions
         User A likes a transaction of User B; User B gets notification that User A liked transaction :
     TypeError: The following error originated from your application code, not from Cypress.

  > Cannot read properties of null (reading 'pulsate')

When Cypress detects uncaught errors originating from your application it will automatically fail the current test.

This behavior is configurable, and you can choose to turn this off by listening to the `uncaught:exception` event.

https://on.cypress.io/uncaught-exception-from-application
      at <unknown> (http://localhost:3000/static/js/vendors~main.chunk.js:363360:25)
      at commitHookEffectListMount (http://localhost:3000/static/js/vendors~main.chunk.js:659730:30)
      at commitPassiveMountOnFiber (http://localhost:3000/static/js/vendors~main.chunk.js:661223:17)
      at commitPassiveMountEffects_complete (http://localhost:3000/static/js/vendors~main.chunk.js:661195:13)
      at commitPassiveMountEffects_begin (http://localhost:3000/static/js/vendors~main.chunk.js:661185:11)
      at commitPassiveMountEffects (http://localhost:3000/static/js/vendors~main.chunk.js:661175:7)
      at flushPassiveEffectsImpl (http://localhost:3000/static/js/vendors~main.chunk.js:663060:7)
      at flushPassiveEffects (http://localhost:3000/static/js/vendors~main.chunk.js:663012:18)
      at commitRootImpl (http://localhost:3000/static/js/vendors~main.chunk.js:662971:9)
      at commitRoot (http://localhost:3000/static/js/vendors~main.chunk.js:662754:9)
      at performSyncWorkOnRoot (http://localhost:3000/static/js/vendors~main.chunk.js:662263:7)
      at flushSyncCallbacks (http://localhost:3000/static/js/vendors~main.chunk.js:650266:26)
      at <unknown> (http://localhost:3000/static/js/vendors~main.chunk.js:661867:17)

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        7                                                                                │
  │ Passing:      6                                                                                │
  │ Failing:      1                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  3                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     2 minutes, 17 seconds                                                            │
  │ Spec Ran:     ui/notifications.spec.ts                                                         │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘

image


SUCCESSFUL RUN

  Running:  ui/notifications.spec.ts                                                      (14 of 21)

  Notifications
    ✓ renders an empty notifications state (5717ms)
    notifications from user interactions
      ✓ User A likes a transaction of User B; User B gets notification that User A liked transaction  (7323ms)
      ✓ User C likes a transaction between User A and User B; User A and User B get notifications that User C liked transaction (6747ms)
      ✓ User A comments on a transaction of User B; User B gets notification that User A commented on their transaction (5130ms)
      ✓ User C comments on a transaction between User A and User B; User A and B get notifications that User C commented on their transaction (9146ms)
      ✓ User A sends a payment to User B (3709ms)
      ✓ User A sends a payment request to User C (3103ms)

  7 passing (41s)

  (Results)

  ┌────────────────────────────────────────────────────────────────────────────────────────────────┐
  │ Tests:        7                                                                                │
  │ Passing:      7                                                                                │
  │ Failing:      0                                                                                │
  │ Pending:      0                                                                                │
  │ Skipped:      0                                                                                │
  │ Screenshots:  0                                                                                │
  │ Video:        true                                                                             │
  │ Duration:     41 seconds                                                                       │
  │ Spec Ran:     ui/notifications.spec.ts                                                         │
  └────────────────────────────────────────────────────────────────────────────────────────────────┘
MikeMcC399 commented 1 year ago

It is also failing on Bitbucket.

See for instance https://bitbucket.org/cypress-io/cypress-realworld-app/pipelines/results/374/steps/%7Be4d08220-ef2f-4d7b-995a-49bfccd649ba%7D

ramosbugs commented 1 year ago

I think this issue is due to the incompatibility between Material-UI 4 and React 18. See https://github.com/mui/material-ui/pull/32395 and https://github.com/mui/material-ui/issues/30953.

MikeMcC399 commented 1 year ago

There is a message in Discord saying that RWA will be reworked, so hopefully this is one of the issues which will get resolved.

MikeMcC399 commented 1 year ago

No current failures. I will re-open if necessary.