Cypress.spec is incorrect when running all specs, should contain actual current spec

[meinaart]

Current behavior:

The contents of Cypress.spec are incorrect when I click on "Run all specs".

It now contains:

{
  absolute: "__all",
  name: "All Specs",
  relative: "__all"
}

I use Cypress.spec in my cypress-plugin-snapshots plugin to determine snapshot filename. I found a very hard work around by using a preprocessor to store the latest processed file. But this breaks other stuff. It would be very useful if Cypress.spec would contain the actual current test.

Desired behavior:

It should contain the details of the spec that is running. Even when you clicked on "Run all specs".

For example:

{
  name: 'filter.spec.js',
  relative: 'cypress/integration/filter.spec.js',
  absolute: '/Users/janelane/Dev/web-app/cypress/integration/filter.spec.js',
}

Steps to reproduce: (app code and test code)

Put code below in a Spec and see contents from Cypress.spec when running with "Run all tests" and by running test directly.

console.log(Cypress.spec);

Versions

All versions that have Cypress.spec.

[jennifer-shehane]

Similar bug with screenshot names: https://github.com/cypress-io/cypress/issues/2319

I believe implementing this would require this feature to be addressed: https://github.com/cypress-io/cypress/issues/1586

[meinaart]

Similar bug with screenshot names: #2319

I believe implementing this would require this feature to be addressed: #1586

I think you are right. Is there an ETA?

[jennifer-shehane]

No

[peter-mw]

Any news on this issue ?

[bahmutov]

@peter-mw in my comment on that issue https://github.com/meinaart/cypress-plugin-snapshots/issues/10#issuecomment-514459554 I show a workaround

[erezrokah]

While not retrieving the same value of spec name (which is the filename), I'm using:

const spec = Cypress.mocha.getRunner().suite.ctx.currentTest.parent.title;
const testName = Cypress.mocha.getRunner().suite.ctx.currentTest.title;

e.g.

describe('some spec', () => {
  it('some test', () => {
    //
  });
})

Will result in spec === "some spec" and testName === "some test".

I can't use the __filename method as I'm importing my tests from another file in my specs (passing them arguments to change the configuration of the tests).

[liamdefty]

👋 I've been battling with this issue too. I came up with a workaround, extending upon @bahmutov's work in https://github.com/meinaart/cypress-plugin-snapshots/issues/10#issuecomment-514459554

In my case, I was copying the integration folders to another location anyway so I modified the file to include a fix for the spec:

const esprima = require('esprima');
const escodegen = require('escodegen');

/**
 * Workaround for incorrect Spec being set in `All Specs`.
 * @see https://github.com/cypress-io/cypress/issues/3090
 *
 * Parses all specs and inserts a before block in each describe block.
 * The before block sets the correct spec file which will be used in all
 * cases. To ensure consistency, we do this every time, even when not using
 * `All Specs`.
 *
 * @param {String} fileContents - Contents of spec file
 * @returns {String} - New spec file with added before block
 */
module.exports = (fileContents) => {
  const beforeBlock = esprima.parseScript(`
    before(() => {
      const path = require('path');
      const relative = __filename.substr(1);
      const integrationFolder = Cypress.config('integrationFolder');
      const absolute = path.join(Cypress.config('projectRoot'), __filename);
      const name = absolute.replace(integrationFolder + '/', '');

      Cypress.spec = { absolute, name, relative };
    });
  `).body[0];

  const parsed = esprima.parseScript(fileContents);

  const parsedFileWithInsert = {
    ...parsed,
    ...parsed.body.map((block) => {
      if (block.expression.callee.name === 'describe') {
        // eslint-disable-next-line no-param-reassign
        block.expression.arguments[1].body.body = [
          beforeBlock,
          ...block.expression.arguments[1].body.body,
        ];
      }
      return block;
    }),
  };

  return escodegen.generate(parsedFileWithInsert);
};

A little hacky, but works for my needs whilst we wait on this issue being fixed. Hope this saves someone the headache I had 😅

[vegerot]

This sounds like a pretty important bug. Does the team here want any assistance with fixing it?

I would like to note that this bug only happens when running in Firefox

when running in Chrome it works fine

[vegerot]

@jennifer-shehane I see you marked this issue with "existing workaround". What is this workaround, exactly?

[vegerot]

Current behavior:

The contents of Cypress.spec are incorrect when I click on "Run all specs".

It now contains:

{
  absolute: "__all",
  name: "All Specs",
  relative: "__all"
}

@meinaart looking into this now. Do you have a specific commit (or at least version) that you first noticed this occurring in?

[JGJP]

@vegerot I believe that the workaround is here

Also, I'm experiencing this in Chrome

[rdadoune]

This solution works for me universally with TypeScript:

support/snapshotPatch.ts:

import { basename } from 'path';

declare global {
  // eslint-disable-next-line @typescript-eslint/no-namespace
  namespace Cypress {
    // eslint-disable-next-line @typescript-eslint/no-unused-vars
    interface Chainable<Subject> {
      /**
       * Patch cypress-plugin-snapshots
       *
       * @returns {void}
       */
      fixCypressSpec(): void;
    }
  }
}

Cypress.Commands.add('fixCypressSpec', function () {
  const { absoluteFile, relativeFile } = this.test.invocationDetails;
  Cypress.spec = {
    ...Cypress.spec,
    absolute: absoluteFile,
    name: basename(absoluteFile),
    relative: relativeFile,
  };
});

support/index.ts:

import './support/snapshotPatch.ts';

beforeEach(() => {
  cy.fixCypressSpec();
});

[develar]

Cypress is great, but I wonder why snapshot testing looks so abandoned. The plugin is not maintained (e.g. https://github.com/meinaart/cypress-plugin-snapshots/issues/186) and Cypress itself contains this bug. I see a workaround, but why it cannot be fixed on Cypress or the plugin side? Two years passed.

I understand, that probably most developers or do not do snapshot testing, or use Jest. But jest + Vue setup is not easy and in any case, I don't need yet another library (moreover, better to use a real browser instead of NodeJS). Because Cypress is awesome. Say, in my case I also want to grab a snapshot after some interaction.

[TD-DO]

I am also having this issue and would like a solution. The workarounds are nice but it would be nicer to not have to use them.

[DiesIrae]

I am having erratic errors with the workaround. Sometimes, test.invocationDetails doesn't exists: Cannot read properties of undefined (reading 'absoluteFile').

It does not happened on the same test every time, so it's quite hard to track. I found no documentation on test.invocationDetails so I'm stuck with re-launching my tests sometimes...

[Yash-Singh1]

Any updates? Not everyone can use the workaround because some people use 3rd party plugins instead of a custom internal plugin.

[Lyokolux]

Cypress is great, but I wonder why snapshot testing looks so abandoned. The plugin is not maintained (e.g. meinaart/cypress-plugin-snapshots#186) and Cypress itself contains this bug. I see a workaround, but why it cannot be fixed on Cypress or the plugin side? Two years passed.

I understand, that probably most developers or do not do snapshot testing, or use Jest. But jest + Vue setup is not easy and in any case, I don't need yet another library (moreover, better to use a real browser instead of NodeJS). Because Cypress is awesome. Say, in my case I also want to grab a snapshot after some interaction.

Any updates on this issue?

[kostiantyn-work]

It seems we need to move to Playwright since Cypress plugins are usually not maintained

[jennifer-shehane]

@kostiantyn-work The plugin referred to in this issue is not maintained or owned by Cypress, it's a community plugin that someone decided to no longer maintain at some point.

We don't have work planned for this issue of fixing what Cypress.spec returns when running all specs. Having this not fully supported is one of the reasons running all specs is an experimental feature.