simonsmith / cypress-image-snapshot

Catch visual regressions in Cypress with jest-image-snapshot
MIT License
64 stars 15 forks source link

Absolute path generated in cypress/snapshot directory #41

Open awnclaim opened 9 months ago

awnclaim commented 9 months ago

Hi there!

With the following configuration options for addMatchImageSnapshotCommand() in index.ts:

  customSnapshotsDir: "cypress/snapshots",
  e2eSpecDir: "cypress/e2e/test-ui-visual",

... and this option passed in cypress.config.ts:

screenshotsFolder: "cypress/snapshots/",

... I didn't expect the final screenshot path to look like this:

111111

Could you please help me determine why so? Thanks in advance!

simonsmith commented 9 months ago

Is there a reason why your screenshots folder is identical to the snapshots folder? That seems like it would cause problems to me. Can you try the Cypress default of cypress/screenshots?

https://docs.cypress.io/guides/references/configuration#Screenshots

awnclaim commented 9 months ago

This is what I get when setting the screenshots folder path to default (the lowest level directory is now "screenshots", not "snapshots", so the original issue is still here):

11111
sbrow commented 9 months ago

I am also experiencing this issue with "default" cypress settings.

Cypress version: 13.2.0 @simonsmith/cypress-image-snapshot: 9.0.1

// cypress.config.js
const { defineConfig } = require("cypress");

const { addMatchImageSnapshotPlugin } = require('@simonsmith/cypress-image-snapshot/plugin');

module.exports = defineConfig({
  //chromeWebSecurity: false,
  e2e: {
    setupNodeEvents(on, config) {
      addMatchImageSnapshotPlugin(on)
    },
  },
});
// cypress/support/e2e.js
// ***********************************************************
// This example support/e2e.js is processed and
// loaded automatically before your test files.
//
// This is a great place to put global configuration and
// behavior that modifies Cypress.
//
// You can change the location of this file or turn off
// automatically serving support files with the
// 'supportFile' configuration option.
//
// You can read more here:
// https://on.cypress.io/configuration
// ***********************************************************

// Import commands.js using ES2015 syntax:
import './commands'

//require('@cypress/snapshot').register()

import { addMatchImageSnapshotCommand } from '@simonsmith/cypress-image-snapshot/command'
addMatchImageSnapshotCommand({
  failureThreshold: 0.005,
  failureThresholdType: 'percent',
})

// Alternatively you can use CommonJS syntax:
// require('./commands')
simonsmith commented 9 months ago

Thanks for the report, I'll take a look this week

@sbrow What platform are you on? The tests in this repository run on Linux/Mac and I see no such issue with default settings

sbrow commented 9 months ago

I am running on NixOS, this is my flake:

{
  description = "Website testing";

  inputs = {
    flake-utils.url = "github:numtide/flake-utils";
  };

  outputs = { self, flake-utils, nixpkgs }:
    flake-utils.lib.eachSystem flake-utils.lib.allSystems
      (system:
        let pkgs = nixpkgs.legacyPackages.${system}; in
        {
          formatter = pkgs.nixpkgs-fmt;

          devShells.default = pkgs.mkShell
            {
              buildInputs = with pkgs; [
                cypress

                nodejs
              ];
            };
        });
}

To make it use the nix bundled version of NixOS, I have have an environment variable set

export CYPRESS_RUN_BINARY=$(which Cypress)

Let me know if you have any other questions. I can provide my flake.lock if needed.

sbrow commented 9 months ago

Looking at #32, I think it's worth noting that the path of my project does include multiple . characters, which may be contributing? /shrug

e.g. /home/sbrow/github.com/testing-website.com/

sbrow commented 8 months ago

New information!

It works fine when I do cypress run, but not when I do cypress ui.

simonsmith commented 8 months ago

Do you mean cypress open?

sbrow commented 8 months ago

yes, my mistake.

awnclaim commented 8 months ago

Same here, guys, with cypress run the path is just fine. image

simonsmith commented 8 months ago

Could you please show me the difference in path between open and run mode that are generated? The screenshots are not that useful for this

Also, note that you can tell Cypress to put open mode snapshots in a different directory if you're not using them (for example if you generate snapshots in run mode via Docker:

const path = Cypress.config('isInteractive')
  ? `cypress/snapshots/${Cypress.browser.name}/open-mode`
  : `cypress/snapshots/${Cypress.browser.name}`;
addMatchImageSnapshotCommand({
  customSnapshotsDir: path,
});
awnclaim commented 8 months ago

Sure, here is the spec path: /Users/diana.basova/Projects/my_project/test/test-ui/cypress/e2e/test-ui-visual/ideaTasks.cy.ts

How the snapshot path looks in run mode:

/Users/diana.basova/Projects/my_project/test/test-ui/cypress/snapshots/ideaTasks.cy.ts/Visual regression tests for Idea Tasks -- Idea Tasks when '390,844' resolution.snap.png

How the snapshot path looks in open mode:

/Users/diana.basova/Projects/my_project/test/test-ui/cypress/snapshots/ideaTasks.cy.ts/Users/dianabasova/Projects/my_project/test/test-ui/cypress/screenshots/Visual regression tests for Idea Tasks -- Idea Tasks when '390,844' resolution.snap.png