meinaart / cypress-plugin-snapshots

Plugin for snapshot tests in Cypress.io
MIT License
497 stars 117 forks source link

Make snapshot file path configurable #9

Open btang-aa opened 5 years ago

btang-aa commented 5 years ago

Hey, it's me again.

The current snapshots are saved in the relative path of the test spec file, under __snapshots__ folder. To improve the performance, we introduce an index_spec.js to collect all test files in the same folder and run them together, now we have 2 issues:

  1. If run with index_spec.js, all snapshots are saved in one index_spec.js.snap, which is not easy to track when we have many cases.
  2. If we just run one spec file, say a_spec.file, the snapshot is generated again a_spec.js.snap in diff path, which will be AUTOPASS because it's not existed before.

I have an idea: generate the file name according to the key word in the test title, but it seems annoying since we need to unify all test titles and index_spec.js is not in common use. So I would like to hear from you, you should have better vision.

Here is my proposal code change(the test title should follow the pattern - [Filename] XXXXXX): function getSnapshotFilename(testFile, snapshotTitle) { const dir = path.join(path.dirname(testFile), DIR_SNAPSHOTS); const filename = `${snapshotTitle.match(/\[(.+)?\]/)[1]}.snap`; return path.join(dir, filename); }

Here is the context of index_spec.js: const reqCont = require.context('./', true, /_spec\.js?$/); reqCont.keys().forEach(filename => reqCont(filename));

And here is our case file structure screenshot: image

renelux commented 5 years ago

@meinaart do you think it is possible to maybe add support for programmatically setting some of the values in the dict that are used to generate the path/filename for the snapshots?

https://github.com/meinaart/cypress-plugin-snapshots/blob/master/src/utils/commands/getTaskData.js#L23

A easy way to do that is to for example modify the config of your plugin dynamically, then if the key exists in the config use that if not use the current behavior.

meinaart commented 5 years ago

I will think about it for a bit. I am sure I can come up with a nice dynamic solution.

Would a filename template work for you? I need to think about what use cases would be valid.

Cypress itself does not recommend combining your test in one big file by the way. Especially not if you want to start using their dashboard service at some moment in time. One big test cannot be run in parallel. This is something I heard via a colleague who spoke with someone who worked at Cypress.

So I am a bit hesitant to fulfilling this request. I understand why you setup your Cypress this way but I think it's discouraged to do it like that. On the other hand I want to make my plugin as flexible as possible. Cypress itself is quite flexible as well.

Things I am considering:

renelux commented 5 years ago

@meinaart we actually have no desire to integrate with their dashboard service, for our use case it is actually quiet overkill. Your colleague is correct that this doesn't work well with their way of running in parallel, but as we don't use the dashboard we can't use their implementation of running in parallel either way :)

So we are not maintaining one big spec file, we actually have around 40 smaller spec files. But we combine them in seperate larger files by importing the whole file in a index_spec. This saved us around 4-5 min on a 15 min total runtime, as Cypress reloads the runner for every spec file.

Just FYI, if you have any sort of implementation that you think will work for you, feel free to highlight it to us and we would be more then happy to help you implement it. We understand that some use cases might be particular to our use case.

meinaart commented 5 years ago

@renelux could you try the latest version and then press "run all" to see if it then behaves if you want? That would not solve your issue completely but would perhaps make it possible to do an easy fix.

ALeschinsky commented 4 years ago

Hello, any updates on this issue? I'm trying to use cypress-plugin-snapshots in an Angular app and it works fine, but the actual screenshots are saved under dist/ folder where the tests are compiled from typescript into javascript. This folder is excluded from git (in .gitignore), so the snapshots are not really part of the repository this way... How can we move the screenshots to where the tests are, so that they are pushed to git and version tracked?

librehat commented 4 years ago

It'll be great if we can make the path configurable so we put the image snapshots in a completely different directory

sharutkarsh commented 1 year ago

Any update here ?