screenshots are not accurately captured when test replay is enabled (with capture: viewport as default with --no-runner-ui) #28073

Open pablosmalys opened 1 year ago

Current behavior

I have set viewport in my cypress config file to 1920x1200, but whenever test fails it sends screenshot in wrong viewports (looks for me that in default values for viewports). Test are running like from local machine via CLI and so from GitLab pipelines. In both situations result is same.

If I disable test replay report, then issue with screenshots disappear.

Desired behavior

The screenshot must be always send in define viewports.

Test code to reproduce

test:

it("visit google", () => {
  cy.visit("https://www.google.com/");
  cy.get(".pavel"); // this will fail, cause there is no such class
});

cypress config:

import { defineConfig } from "cypress";

export default defineConfig({
  e2e: {
    viewportWidth: 1920,
    viewportHeight: 1200,
    chromeWebSecurity: false,
    requestTimeout: 30000,
    responseTimeout: 30000,
    experimentalRunAllSpecs: true,
    projectId: "", //put your projectId 
    setupNodeEvents(on, config) {
      return require("./cypress/plugins/index.js")(on, config);
    },
  },
  // video: true,
  // retries: 1,
});

Cypress Version

13.3.1

Node version

v16.20.0

Operating System

macOS 13.6

Debug Logs

No response

Other

No response

When you --record and have Test Replay enabled, starting in v13 we remove the Command Log from the UI and attempt to take the screenshots, we default use cy.screenshot({ capture: viewport }).

Likely this is highlighting a bug with our capture: viewport option, where it does not capture properly (Perhaps this one). I'll leave this bug report open.

If you are recording to the Cloud, I would recommend referring to Test Replay for the representation of your App at the time it ran. If screenshots are still important to be captured, I would recommend passing the --runner-ui flag when running tests. This use the behavior you're seeing when you don't record. It will however render the command log during runs and it will show up in the screenshot, which we did remove in v13 to improve performance.

hi @jennifer-shehane, thank you for your respond.

Just to clarify what I am reporting, it is not as much about runner UI (yes I do not need runner ui on my failure screenshots) as about that screenshot of my app is taken in incorrect viewports after failure. Yes, we also need to have screenshots in our cypress cloud if test failed, but of course we use Test replay most of the time.

So in first screen you can not see even all buttons of app, because viewports is definitely not 1920x1200. But from second screen I am expecting this part to be whole captured and send to cloud:

I hope I explained properly :) Let me know if you have further questions

Hi @jennifer-shehane and @Cypress.

Since upgrading to Cypress 13 is so strongly recommended and this is a regression in the previous behavior, please set this bug with higher priority. I agree there is a workaround but that one does not fully solve all our issues.

Please update on the current status or future plans for this bugfix. Thank you.

@cornelcgm Thanks for your input. We'll look to see if we can prioritize this with other work in the upcoming quarter. Not sure I can promise it though.

This issue has not had any activity in 180 days. Cypress evolves quickly and the reported behavior should be tested on the latest version of Cypress to verify the behavior is still occurring. It will be closed in 14 days if no updates are provided.

this bug still reproducing, commenting so it won't be closed

cypress-io / cypress