localvoid / chai-karma-snapshot

Chai Plugin for Snapshot Testing with Karma
MIT License
4 stars 8 forks source link

Doesn't work with non-primitive values #3

Open dentuzhik opened 7 years ago

dentuzhik commented 7 years ago

I'm pretty sure you know that, but just for others who'll stumble upon it. This plugin currently doesn't work with non-primitives because of strict equality comparison in assertion: https://github.com/localvoid/chai-karma-snapshot/blob/master/src/index.ts#L38

Easy way would be to fix it with deep equality comparison, but it would make sense to maybe use some other implementation of comparison altogether, so it will support nice diffs with different types of serializable values (such as JSX).

I've done quick research and apart from packages which jest exposes, there're two more or less similar implementations, maybe it makes sense to peek snapshots comparison from there.

https://github.com/facebook/jest/blob/master/packages/jest-snapshot https://github.com/bahmutov/snap-shot https://github.com/suchipi/chai-jest-snapshot

Do you have any ideas regarding that? I'm interested to make snapshot testing using karma in my project, could help with some research / PRs.

localvoid commented 7 years ago

I think that Jest just automatically serializes values to strings[1] with pretty-format package[2] and then using strict equality[3] and then reporter performs a string diffing.

This package doesn't have automatic serialization, but Mocha reporter should perform string diffing when it is enabled mochaReporter: { showDiff: true }. Maybe just adding automatic serialization will be enough?

  1. https://github.com/facebook/jest/blob/ffd43f7338125d987b154cb91d98251a37c7e5bd/packages/jest-snapshot/src/State.js#L118
  2. https://github.com/facebook/jest/blob/master/packages/pretty-format/src/plugins/react_element.js
  3. https://github.com/facebook/jest/blob/ffd43f7338125d987b154cb91d98251a37c7e5bd/packages/jest-snapshot/src/State.js#L120
localvoid commented 7 years ago

I can merge this branch[1] with jest serialization and publish, I hope that everything will work in browser. Tried with basic React elements and it seems that it is working.

https://github.com/localvoid/chai-karma-snapshot/tree/jest-serialization

dentuzhik commented 7 years ago

I'll checkout the branch, and try on my setup with enzyme wrappers, and let you know.

dentuzhik commented 7 years ago

Couldn't make it work through npm link for whatever reason, if installing through npm install can't build it, because branch isn't rebased on top of master and files are missing.

localvoid commented 7 years ago

Published 0.2.0 with jest serialization.

dentuzhik commented 7 years ago

Ok, it seems to work nicely with enzyme and enzyme-to-json. The problem though, that snapshot is almost entirely unreadable, because it's compiled to the JS string, what makes it impossible to review.

Would be nice emulate the jest behaviour and export to template strings. Also side question, where should I look for implementation of the writing to the snapshot file? I planned to use it for react components, and would be nice to store snapshots alongside actual components, instead of single location, so was wondering if it's possible to enable/configure such a behaviour.

dentuzhik commented 7 years ago

Found it in: https://github.com/localvoid/karma-snapshot/blob/master/lib/index.js

localvoid commented 7 years ago

Would be nice emulate the jest behaviour and export to template strings.

Yes, I think that it should be implemented as a middleware that automatically converts from readable format that is stored on disk to something that will work across all browsers. I'll try to implement it right now.

I planned to use it for react components, and would be nice to store snapshots alongside actual components, instead of single location, so was wondering if it's possible to enable/configure such a behaviour.

Yes, agree, but I have no idea how to retrieve file names. Maybe inject something with a webpack plugin.

localvoid commented 7 years ago

I think that it also will be great to store snapshots as a markdown file. It would allow automatic syntax highlighting for snapshots in editors that support markdown (for ex. VSCode).

dentuzhik commented 7 years ago

Yes, agree, but I have no idea how to retrieve file names. Maybe inject something with a webpack plugin.

Found this. There's a chance that it should be exposed under result.suite.file, I'll try to check it locally.

dentuzhik commented 7 years ago

Can you maybe specify some instructions how to develop these packages locally (and test them)? So far all my attempts to link packages into existing testing setup fail with errors.

localvoid commented 7 years ago

They don't have any build steps, so simple git clone, npm install, npm link should work.

This package is written in TypeScript, so it should be compiled with npm run dist.

Basic test project

{
  "private": true,
  "name": "chai-test",
  "scripts": {
    "test": "karma start --single-run"
  },
  "devDependencies": {
    "babel-core": "^6.25.0",
    "babel-runtime": "^6.23.0",
    "chai": "^4.1.0",
    "chai-karma-snapshot": "^0.1.2",
    "corejs": "^1.0.0",
    "karma": "^1.7.0",
    "karma-chrome-launcher": "^2.2.0",
    "karma-mocha": "^1.3.0",
    "karma-mocha-reporter": "^2.2.3",
    "karma-mocha-snapshot": "^0.2.0",
    "karma-snapshot": "^0.1.0",
    "karma-sourcemap-loader": "^0.3.7",
    "karma-webpack": "^2.0.4",
    "mocha": "^3.4.2",
    "react": "^15.6.1",
    "react-dom": "^15.6.1",
    "webpack": "^3.3.0"
  }
}
const webpack = require("webpack");

module.exports = function (config) {
  config.set({
    browsers: ["ChromeHeadless"],
    frameworks: ["mocha", "snapshot", "mocha-snapshot"],
    reporters: ["mocha"],
    preprocessors: { "__tests__/index.js": ["webpack", "sourcemap"] },
    files: ["__tests__/index.js"],

    colors: true,
    autoWatch: true,

    webpack: {
      plugins: [
        new webpack.SourceMapDevToolPlugin({
          test: /\.js$/,
        }),
      ],
      performance: {
        hints: false
      },
    },

    webpackMiddleware: {
      stats: "errors-only",
      noInfo: true
    },

    snapshot: {
      update: !!process.env.UPDATE,
    },

    mochaReporter: {
      showDiff: true,
    },

    client: {
      mocha: {
        reporter: "html",
        ui: "bdd",
      }
    },
  });
};
import "core-js";
import { use, expect } from "chai";
import { matchSnapshot } from "chai-karma-snapshot";
import * as React from "react";
use(matchSnapshot);

it("check snapshot", () => {
  expect("Hello World").to.matchSnapshot();
});

it("object", () => {
  expect({ a: "abc" }).to.matchSnapshot();
});

it("React", () => {
  expect(React.createElement("div", {}, React.createElement("span"))).to.matchSnapshot();
});
$ npm install
$ npm link karma-snapshot
$ npm link karma-mocha-snapshot
$ npm link chai-karma-snapshot
$ npm run test
localvoid commented 7 years ago

Published new versions for karma-snapshot and chai-karma-snapshot. Snapshots will be stored in a markdown format like this:

## `Root Suite`

##   `Sub Suite`

##     `Sub Sub Suite`

####       `object`

```js
Object {
  "a": "abc",
}

React

<div>
  <span />
</div>

- H2 is used for suites, H4 for tests
- suite names and test names are wrapped in inline code blocks to prevent any conflicts with markdown format, it also checks for backticks and tries to find a safe delimiters.
- because suites can have any depth, instead of relying on different headers, I've chosen to use offsets to detect correct depth.
- when code language is specified, it will be added to snapshot code blocks (enables syntax highlighting in code editors)

### `matchSnapshot()` changes

It now has interface: `matchSnapshot(lang?: string, update?: boolean)`.

`lang` is used to specify code language and `update` will force updating snapshot.

### Preprocessor

Snapshot preprocessor should be added to Karma configs:

```js
config.set({
  preprocessors: {
    "**/__snapshot__/**/*.md": ["snapshot"],
    ...
  }
});
localvoid commented 7 years ago

I have an idea how to store snapshots in different files, but it is an ugly workaround :) We can just specify file path in a root suite:

describe("src/components/MyComponent", () => {
  it("test");
});
localvoid commented 7 years ago

Published new versions. It is now extracting snapshot file path from the name of root suites. Also, unique names for root suites will solve problem with possible name collisions because all tests are running in the same namespace.

Karma Config update

Snapshots should be added to files directive:

config.set({
  files: [
    "**/__snapshots__/**/*.md",
    ...
  ]
});

Autowatch Mode

Don't know how to fix this, but right now it may trigger test rerun when snapshots are updated because Karma is watching snapshot files.