Proposal: add snapshot testing (this is not screenshot diffing)

[bahmutov]

Similar to Jest / Ava snapshot testing (see https://glebbahmutov.com/blog/snapshot-testing/ and https://glebbahmutov.com/blog/picking-snapshot-library/) it would be nice to save / compare values to snapshots.

For example this test can save the wrapped objects on the first run, then we would check in the generated snapshots.js file. Next time Cypress runs it would compare new values with saved snapshots and throw an error if they are different.

Here is a proof of concept

describe('my tests', () => {
  it('works', () => {
    cy.log('first snapshot')
    cy.wrap({foo: 42}).snapshot()

    cy.log('second snapshot')
    cy.wrap({bar: 101}).snapshot()
  })
})

The test runner

When clicking on "snapshot" in the reporter, it shows the object in the dev tools

Similarly, if previous command yields a jQuery element, then it is converted into JSON and saved. For example, we could snapshot focused element before and after typing (TodoMVC example at todomvc.com/examples/react/)

describe('focused input field', () => {
  it('is empty and then typed into', () => {
    cy.visit('http://todomvc.com/examples/react/')
    cy.focused()
      .snapshot('initial') // given human names as well
      .type('eat healthy breakfast')
      .snapshot('after typing')
  })
})

When clicking on the snapshot in this case, prints original element and JSON object

The snapshots are all loaded at the start and saved together at the end. Here is snapshot file

module.exports = {
  "my tests": {
    "works": {
      "1": {
        "foo": 42
      },
      "2": {
        "bar": 101
      }
    }
  },
  "focused input field": {
    "is empty and then typed into": {
      "initial": {
        "tagName": "input",
        "attributes": {
          "class": "new-todo",
          "placeholder": "What needs to be done?",
          "value": ""
        }
      },
      "after typing": {
        "tagName": "input",
        "attributes": {
          "class": "new-todo",
          "placeholder": "What needs to be done?",
          "value": "eat healthy breakfast"
        }
      }
    }
  },
  "__version": "1.0.2"
}

If a run time value during test does not match value loaded from snapshot, it throws an exception. For example, changed code to cy.wrap({foo: 45}).snapshot() leads to

[brian-mann]

Loving this. Can we also add it for elements?

// should serialize the HTML as a string and inline 
// certain things like value, checked, selected, etc
cy.get('form').snapshot() 

Bonus points if you format the HTML nicely 👍

[bahmutov]

It is already there - the focused().snapshot() is an example. Instead of HTML it converts DOM to json though

Sent from my iPhone

On Oct 19, 2017, at 13:18, Brian Mann notifications@github.com wrote:

Loving this. Can we also add it for elements?

cy.get('form').snapshot() // should serialize the HTML as a string and inline certain things like value, checked, selected, etc Bonus points if you format the HTML nicely 👍

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

[brian-mann]

I see... is this what jest does as opposed to string HTML?

[bahmutov]

I think Jest converts to HTML, but I am not sure. But I think this is great user space feature, so I can make this public under @bahmutov/cypress-snapshot, see how it goes and then we can move into @cypress/<..> or build into Cypress itself. Plus we can have options how to stringify values before comparing (already snapshot supports it under the hood), which is something Jest does too.

[brian-mann]

I want to see this in cypress core... it's incredibly useful and powerful for a myriad of things.

Using the console as a comparison is super nice to explore the object structures.

We could even potentially compare the HTML using the actual application under test by inlining all of the styles or capturing them as well separately and applying the styles to the elements.

[bahmutov]

Yes!!!

Sent from my iPhone

On Oct 19, 2017, at 13:34, Brian Mann notifications@github.com wrote:

I want to see this in cypress core... it's incredibly useful and powerful for a myriad of things.

Using the console as a comparison is super nice to explore the object structures.

We could even potentially compare the HTML using the actual application under test by inlining all of the styles or capturing them as well separately and applying the styles to the elements.

— You are receiving this because you were assigned. Reply to this email directly, view it on GitHub, or mute the thread.

[jennifer-shehane]

Just wanted to bring up that we use the terminology 'snapshot' for explaining the 'time-travel' process of restoring the DOM when hovering over commands and 'pinning' - we will have to rename this to something else, since I think most people define 'snapshots' as the feature explained in this proposal.

[bondz]

Yes @jennifer-shehane. New to Cypress, and every time I read snapshot in the docs, I thought it had support for jest snapshots, which I thought was awesome. Seeing that is uses chai is what led me here.

[newtack]

We use Jest for the rest of our testing, so if Cypress could integrate with Jest that would be awesome.

[vinayakkulkarni]

We use Ava across our applications. Any Cy + ava examples?

[gsouf]

Interestingly there doesn't seem to have any good solution for json/data snapshots that is maintained for cypress 10+ and cypress mentioned 5 years ago that it's a plan to get it to the core https://www.cypress.io/blog/2018/01/16/end-to-end-snapshot-testing/.

[fcuenya]

Any progress on this, or if it has been replaced with a 3rd party plugin or a different approach, what would that be?

I want to incorporate this testing approach to my project and I was wondering what would be the latest best practices around it.