percy / percy-cypress

Visual testing with Cypress and Percy
https://percy.io
MIT License
340 stars 41 forks source link

Uncaught error was detected outside of a test: No commands were issued in this test #679

Closed floroz closed 1 year ago

floroz commented 1 year ago

The problem

Our Percy tests in Cypress are failing intermittently with the below error:

An uncaught error was detected outside of a test (failed)

https://github.com/percy/percy-cypress/assets/45793869/bd3dbb1c-14c6-478a-b9a2-2da9f93d1f56

This happens intermittently in random tests, never in the same twice, and only happens when running the test using Cypress.

We run using the following command:

percy exec -- cypress run --headless --browser electron --component --config excludeSpecPattern='!(**/visuals/**)'

and our setup is the same for all tests.

it("Percy: Visual Tests", () => {
  cy.mount(AllVariants); // AllVariants is function that returns an array of JSX Vue Components

  // Always validate elements are visible before triggering Percy Snapshots.
  cy.get(textSelector).first().should("have.text", label);

  cy.percy("<BCheckbox />", "desktop");
});

cy.percy is a wrapper that we created as a custom command:

Cypress.Commands.add(
  "percy",
  (name: PercyName, deviceUnderTest: DeviceUnderTest) => {
    const deviceMap = {
      "desktop": [1280],
      "mobile": [375],
      "desktop-and-mobile": [375, 1280],
    };
    cy.percySnapshot(name, {
      widths: deviceMap[deviceUnderTest] ?? deviceMap["desktop"],
    });
  }
);

We have separate cypress integration for the same components where we don't experience this error.

Environment

floroz commented 1 year ago

Seems related to https://github.com/cypress-io/cypress/issues/25913

itsjwala commented 1 year ago

hey @floroz 👋

have you tried running without percy exec --?

just the following part, if it still breaks, then this is not related to Percy. cypress run --headless --browser electron --component --config excludeSpecPattern='!(**/visuals/**)'

floroz commented 1 year ago

hey @floroz 👋

have you tried running without percy exec --?

just the following part, if it still breaks, then this is not related to Percy. cypress run --headless --browser electron --component --config excludeSpecPattern='!(**/visuals/**)'

Hi @itsjwala, thanks for the response.

The bug seems related to https://github.com/cypress-io/cypress/issues/25913 and it appears quite nasty to solve.

We are migrating our Percy tests away from Component Testing and using Cypress E2E to avoid loading components as async chunk.

So far it looked promising.

I'll close the issue since it's not related to Percy SDK.