[Suggestion] .toVerifyAllCombinations([args]) matcher for Functions

Feature Request

Hi there 👋

Description

I'd like to propose a new Function matcher: .toVerifyAllCombinations([args]).

It would test all combinations of given parameters to the function under test, and match against previous snapshot.

The main usage would be a test-after scenario (it's a snapshot test), to quickly set up a safety net over legacy code, before refactoring.

📖 Context: "Approval testing" of legacy code

Sometimes you don’t know what a piece of legacy code precisely does. But you do know it works in production. If you want to touch this piece of code, you'd better put some tests on it.

One approach to use in this situation is called "Approval testing". It can get you test coverage quickly without having to understand the code.

The recipe is the following:

Determine the different inputs you could combine to test your code
Determine the output of the code you could make a snapshot from
Use test coverage to determine which parts of the code are not covered yet
Augment your inputs combination until you cover all of the code
Perform little mutations in your covered code to check the quality of your snapshot
Augment your inputs combination until there is no way you can add mutations without the test failing

In the end, you got a snapshot of what your code does. And you can start refactoring with confidence.

Further information:

Approval tests: http://approvaltests.com/
Emily Bache presenting the technique on Gilded Rose refactoring kata: https://youtu.be/zyM2Ep28ED8

📽 Usage examples

expect(myFunction).toVerifyAllCombinations([args]);

myFunction would be the function to test, or a wrapper around it which returns a value we can snapshot.

[args] would be list of values to combine, for each argument of myFunction.

UX would be the one of using Jest's .toMatchSnapshot().

A theoretical example

function myFunction(aNumber, aString) {
  if (aNumber > 0) {
    return `${aString} #${aNumber}`;
  }

  if (aString === "foo") {
    return `${aNumber} bar`;
  }

  return "This is twisted…";
}

it("should continue working as before", () => {
  expect(myFunction).toVerifyAllCombinations([1, -1], ["random", "foo"]);
});

It will test all following combinations:

myFunction(1, "random") => random #1
myFunction(1, "foo") => foo #1
myFunction(-1, "random") => This is twisted…
myFunction(-1, "foo") => -1 bar

And produce a snapshot of this, so we can ensure every path is tested.

Another, concrete example

Using Emily Bache's example of testing Gilded Rose behaviour, final test would look like that:

import { Item, Shop } from "./gilded-rose";

describe("Gilded Rose", () => {
  it("should update quality", () => {
    expect(doUpdateQuality).toVerifyAllCombinations(
      [
        "foo",
        "Aged Brie",
        "Backstage passes to a TAFKAL80ETC concert",
        "Sulfuras, Hand of Ragnaros"
      ],
      [-1, 0, 2, 6, 11],
      [0, 1, 49, 50]
    );
  });

  function doUpdateQuality(name, sellIn, quality) {
    const gildedRose = new Shop([new Item(name, sellIn, quality)]);
    const items = gildedRose.updateQuality();
    return items[0];
  }
});

And the produced snapshot:

// Jest Snapshot v1, https://goo.gl/fbAQLP

exports[`Gilded Rose should update quality 1`] = `
Object {
  "Aged Brie,-1,0": Item {
    "name": "Aged Brie",
    "quality": 2,
    "sellIn": -2,
  },
  "Aged Brie,-1,1": Item {
    "name": "Aged Brie",
    "quality": 3,
    "sellIn": -2,
  },
  "Aged Brie,-1,49": Item {
    "name": "Aged Brie",
    "quality": 50,
    "sellIn": -2,
  },

  //… all possible combinations of "Aged Brie"

  "Backstage passes to a TAFKAL80ETC concert,-1,0": Item {
    "name": "Backstage passes to a TAFKAL80ETC concert",
    "quality": 0,
    "sellIn": -2,
  },
  "Backstage passes to a TAFKAL80ETC concert,-1,1": Item {
    "name": "Backstage passes to a TAFKAL80ETC concert",
    "quality": 0,
    "sellIn": -2,
  },

  //… you get the idea
}
`;

Possible solution

Implementation would look like this:

import { toMatchSnapshot } from "jest-snapshot";
import { getAllCombinations } from "./get-all-combinations"; // to implement

function toVerifyAllCombinations(fn, ...args) {
  const snapshot = {};

  getAllCombinations(args).forEach(combination => {
    snapshot[combination] = fn(...combination);
  });

  return toMatchSnapshot.call(this, snapshot);
}

I already implemented the matcher and was about to make a package so I could use it. But I thought it might worth proposing to enhance the list of jest-extended matchers 💡

What are your feelings around this?

If you think it's a good idea, I can push a PR to implement.

jest-community / jest-extended