percy / percy-cypress

Visual testing with Cypress and Percy
https://percy.io
MIT License
340 stars 41 forks source link

Images not captured by percy with cypress v13.6.4 #751

Closed esetnik closed 6 months ago

esetnik commented 7 months ago

The problem

Snapshots are no longer capturing some assets as of cypress v13.6.4. It may be related to changes in https://github.com/cypress-io/cypress/issues/28747

Environment

Details

Every snapshot has visual snapshot changes when upgrading cypress to v13.6.4 from v13.6.3. It looks like any svg icon is not captured by percy snapshots and appear as a gray box instead of the original icon. For example:

v13.6.3

Screenshot 2024-02-05 at 9 06 09 AM

v13.6.4+

Screenshot 2024-02-05 at 9 06 15 AM
github-actions[bot] commented 7 months ago

This issue is stale because it has been open for more than 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.

esetnik commented 7 months ago

Remove stale

ninadbstack commented 7 months ago

@esetnik Thanks for opening the issue here for icon assets not loading. We have also received the support ticket for the same, we are looking into this.

Note: You will receive most of the updates on the support ticket for the same.

github-actions[bot] commented 6 months ago

This issue is stale because it has been open for more than 14 days with no activity. Remove stale label or comment or this will be closed in 14 days.

esetnik commented 6 months ago

remove stale

esetnik commented 6 months ago

Also can confirm this is still happening in the newly released cypress v13.7.0

ninadbstack commented 6 months ago

@esetnik we attempted same using your production website and newer version of cypress and we were unable to reproduce this as replied on the support ticket. We continue to see icons correctly in newer versions of cypress as well.

Example code that we tested with

describe('Test', function() {
  it('Logs in with valid credentials', () => {
    // Visit the login page
    cy.visit('https://frontend.***.com/login');

    // Input username
    cy.get('input[name="email"]').type(email);

    // Input password
    cy.get('input[name="password"]').type(password);

    // Click on the login button
    cy.get('button[type="submit"]').click()

    cy.wait(5000);
    cy.percySnapshot()
  })
})

Versions that we attempted to reproduce with @percy/client/1.27.7 @percy/cli-exec/1.27.7 @percy/cypress/3.1.2 (node/v20.11.0; cypress/13.6.6)

similarly we also tried with 13.6.4 and 13.6.5.

It would be great if you can provide use with reproducible example. [ Or try above example to see if you can reproduce it ]

esetnik commented 6 months ago

I have responded to your email with detailed debug logs from both 13.6.3 and 13.6.4 and there's an obvious difference in asset discovery logs where all of the svg assets are not discovered in 13.6.4. Please review my findings and let me know how to solve this.

ninadbstack commented 6 months ago

We have released a beta version of @percy/cli for the same 1.28.3-beta.0 this was not directly related to cypress, but indirectly related as cypress upgrade updated internal chrome version from 119 to 121 - where on 119 we needed --webkit-mask css property and on 121 onwards mask is supported. So basically mask was being used in discovery stage of percy which did not support it as it was on older chromium version.

esetnik commented 6 months ago

I am testing with 1.28.3-beta.0 now and will let you know if it solves the issue. Thanks!

esetnik commented 6 months ago

@ninadbstack I can confirm the issue is fixed in 1.28.3-beta.0.

ninadbstack commented 6 months ago

This will be available in stable release on 12th of April. Till then feel free to use the beta version.