cypress-io / cypress

Fast, easy and reliable testing for anything that runs in a browser.
https://cypress.io
MIT License
46.72k stars 3.16k forks source link

Cypress version sometimes doesn't load in header #21897

Closed sync-by-unito[bot] closed 1 year ago

sync-by-unito[bot] commented 2 years ago

Current Behavior

These are screenshots from Percy that appear to show that sometimes, the version running is not populated in the launchpad header. These are a source of flake in the Percy screenshots, so it is not consistent between runs.

Screen Shot 2022-05-09 at 10 51 32 AM

Screen Shot 2022-05-09 at 10 51 25 AM

(Percy screenshot )

I also found a flaky test where this would seem to be the explanation:

Screen Shot 2022-05-26 at 9 02 17 AM

Desired Behavior

In this situation, Cypress should always show the current version in the header.

Reproducible Steps

I have not been able to reproduce this consistently but here is a loom showing the issue - required a refresh to show the version in the header. I think it represents a bug because we should not render this part of the UI without the version information, so possibly we sometimes render too early, and need to either expand our v-if conditions for rendering, or look at something else around how this data is resolved, since it involves network calls. At a glance, those calls appear to be awaited so it’s not totally obvious what would make this flaky. If it’s a matter of timing in the tests that take the snapshot, and this info does always eventually populate, we should update our tests.

https://user-images.githubusercontent.com/1271364/171287284-8c2e9698-db9e-4a94-9206-b18e4b2610f1.mp4

┆Issue is synchronized with this Jira Bug by Unito ┆Attachments: Screen Shot 2022-05-09 at 10.51.25 AM.png | Screen Shot 2022-05-09 at 10.51.32 AM.png | Screen Shot 2022-05-26 at 9.02.17 AM.png ┆author: Mark Noonan ┆epic: Minor Bugs ┆friendlyId: UNIFY-1738 ┆priority: High ┆sprint: Fast Follows 1 ┆taskType: Bug

sync-by-unito[bot] commented 2 years ago

➤ Tyler Biethman commented:

This is reproducible in any of our e2e launchpad tests in open mode. Frequently they will render without the version; reloading the frame in the browser will always cause it to be shown.

The versions field is marked with deferIfNotLoaded on the base Query. Making this a standard async field results in the version loading consistently in tests, but it’s slower. We might just be missing an update emission somewhere for the deferred response.

sync-by-unito[bot] commented 2 years ago

➤ Tyler Biethman commented:

I was able to track the test flake down to the launchpad’s pre-mount query we execute here https://github.com/cypress-io/cypress/blob/develop/packages/launchpad/src/main.ts#L30-L32. Removing that query, or even just removing the toPromise() call from it, results in consistent test output without a noticeable increase in load time, at least on my device. Seems like there’s a race condition somewhere in the client and the plugin we use for deferred field resolution? When the version doesn’t appear, we don’t receive a pushFragment subscription change on the client, but we do send it on the server.

While we’re probably safe to remove this query to fix the flake, it’d be worth figuring out exactly why this is happening so we don’t run into similar problems going forward.

marktnoonan commented 2 years ago

Noticed what I think is another issue caused by a similar thing. It doesn't show the release date for the new version here:

missing release date


But it did after a refresh:

has release date, after refresh

tbiethman commented 2 years ago

https://github.com/cypress-io/cypress/pull/21982 was merged to address this, but it definitely seems to still be occurring.

rachelruderman commented 2 years ago

Note: refactoring Promise.all in https://github.com/cypress-io/cypress/blob/develop/packages/launchpad/src/main.ts#L23 seemed to greatly reduce the flake -- for me it eliminated flake locally, but it still occurred very rarely in CI. However since it didn't fix it, I didn't commit the change

makeUrqlClient({ target: 'launchpad' })
.then((launchpadClient) => {
  app.use(urql, launchpadClient)

  // Loading the Header Bar Query document prior to mounting leads to a better experience
  // when doing things like taking snapshots of the DOM during testing, and it
  // shouldn't be any different to the user
  launchpadClient
  .query(HeaderBar_HeaderBarQueryDocument)
  .toPromise()
  // Make sure highlighter is initialized immediately at app
  // start, so it's available when we render code blocks
  .then(() => initHighlighter().then(() => app.mount('#app')))
})
github-actions[bot] commented 1 year ago

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

cypress-app-bot commented 1 year ago

This issue has been closed due to inactivity.