axross / tap-diff

:dancers: The most human-friendly TAP reporter.
MIT License
94 stars 26 forks source link

Incorrect Diffing #17

Open inadarei opened 6 years ago

inadarei commented 6 years ago

For most but very simple diffs the output is simply incorrect. The root problem seems to be that in JSON the order of elements in the object is insignificant, but the diff used in tap-diff doesn't understand that.

Let's look at this example:

var test = require('tape');

test('tape-diff test', function (t) {
    const expected = {
      "one" : "val",
      "two" : "val",
      "three" : "val",
      "four" : "val",
    };

    const actual = {
      "four" : "val",
      "three" : "val",
      "two" : "val",
    };

    t.deepEquals(actual, expected);
    t.end();
});

The proper diff output here is identifying the single missing line:

image

which is what http://www.jsondiff.com/ does.

However, tap-diff doesn't know that the order of elements is insignificant and goes bananas with the diff, producing something akin to:

image

which is pretty useless. And this is an extremely simple example. With a very large and complicated JSON the problem makes tap-diff virtually useless.

Solution

start using the same semantic comparison library that http://www.jsondiff.com/ does to fix the problem. It is: https://github.com/zgrossbart/jdd

felamaslen commented 5 years ago

This issue is caused by the objects being treated, and compared, as strings. PR 20 fixes it.