Print the compared data in the report

[foaw]

Is your feature request related to a problem? Please describe. In my experience, it's so happened that I mostly send off big JSONs, looking all undistinguishably from one another. To even tell where an assertion has failed, or rather on what data, I added a custom message. Then I go to the place where the data is defined and look for it.

The last part, or in some cases even writing assertion descriptions, can be avoided. If I had just the full JSONs of compared data, it would already be enough sometimes.

Describe the solution you'd like. The data compared can be included in the exception message at the top, as is the case with structural differences. In their messages, they have the expected and the actual data, but only rooted at the level where there were unmatching keys or values.

Describe alternatives you've considered. None. I don't think there are any. I looked in the source code, and it seems like no functionality is provided for this, either via SPI, options, or any other levers.

[lukas-krecan]

Hi, can you please share some example, where the error message is not sufficient?

[foaw]

Hi there! I wouldn't call it insufficient, as it gives exactly what isn't adding up—all paths where the values are different, and the values themselves at the paths.

The thing is, from "X is Y; expected Z" I can't figure out why. This is where I need the entire JSON at hand, from which I could learn object I'm dealing with, what identifier it has, what other properties, etc.

At the moment, the best alternative I was able to come up with, is to include some means of identification into the assertion messages, which isn't ideal though. Although this way I can find the expected JSON, there is still no information as to what the actual value turned out to be.

[foaw]

So it's not insufficient for comparing data, where it does a great job at giving you where the values have diverged. It would just be handy if this was included too. I can see that not all people may want to have this behaviour by default, and so I'm only suggesting to provide the feature and give the ability to opt-in to this, without making it the default.

---

By the way, after filing this issue, I though it'd also be cool if the stacktrace wasn't repeated for every difference, as it's the same stacktrace anyway. Should I open another issue for this, or can these two be merged, if we say that this one isn't about printing compared data, but changing the report format in general?

[lukas-krecan]

1. If you add opentest4j library to your classpath, JsonUnit will throw AssertionFailedError which will contain the actual and expected values. You can catch the exception and do whatever you need.
2. About the stack traces - it depends on how are you using the library, I think it prints a stack-trace per exception. Can you please share some code?

[foaw]

Looked once again into the code. I guess I wasn't as attentive yesterday—it does already print what I'm proposing in net.javacrumbs.jsonunit.core.internal.Diff#logDifferences.

https://github.com/lukas-krecan/JsonUnit/blob/b875c5d6a95a36b36d05e7064e571690f42d8e64/json-unit-core/src/main/java/net/javacrumbs/jsonunit/core/internal/Diff.java#L563-L572

Although it only does so at the DEBUG level and unformatted. Also, it also doesn't follow the format IntelliJ IDEA expects, so it's kind of hard to notice, much less compare with the unaided eye.

A sample to illustrate the issue.

```java public class Demo { @Test void unrelatedStructures() { String actual = """ { "foo": { "bar": "baz" }, "bat": { "quux": "qwe" } } """; String expected = """ { "quux": { "qwe": "foo" }, "bat": { "baz": "bar" } } """; verifyJson(actual, expected); } @Test void differentInNested() { String actual = """ { "root": { "foo": "bar" }, "a": { "b": "c" } } """; String expected = """ { "root": { "bar": "foo" }, "a": { "c": "b" } } """; verifyJson(actual, expected); } private static void verifyJson(@Language("json") String actual, @Language("json") String expected) { assertThatJson(actual).isEqualTo(expected); } } ```

Output of differentInNested.

``` org.opentest4j.AssertionFailedError: Different keys found in node "a", missing: "a.c", extra: "a.b", Expected :{"c":"b"} Actual :{"b":"c"} at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError.getError(ExceptionFactory.java:55) <7 internal lines> at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError.(ExceptionFactory.java:46) at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory.createException(ExceptionFactory.java:37) at net.javacrumbs.jsonunit.core.internal.ExceptionUtils.createException(ExceptionUtils.java:45) at net.javacrumbs.jsonunit.core.internal.Diff.failIfDifferent(Diff.java:613) at net.javacrumbs.jsonunit.assertj.JsonAssert.isEqualTo(JsonAssert.java:121) at demo.Demo.verifyJson(Demo.java:57) at demo.Demo.differentInNested(Demo.java:50) <31 internal lines> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) <9 internal lines> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) <28 internal lines> org.opentest4j.AssertionFailedError: Different keys found in node "root", missing: "root.bar", extra: "root.foo", Expected :{"bar":"foo"} Actual :{"foo":"bar"} at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError.getError(ExceptionFactory.java:55) <7 internal lines> at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError.(ExceptionFactory.java:46) at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory.createException(ExceptionFactory.java:37) at net.javacrumbs.jsonunit.core.internal.ExceptionUtils.createException(ExceptionUtils.java:45) at net.javacrumbs.jsonunit.core.internal.Diff.failIfDifferent(Diff.java:613) at net.javacrumbs.jsonunit.assertj.JsonAssert.isEqualTo(JsonAssert.java:121) at demo.Demo.verifyJson(Demo.java:57) at demo.Demo.differentInNested(Demo.java:50) <31 internal lines> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) <9 internal lines> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) <28 internal lines> net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError: JSON documents are different: Different keys found in node "a", missing: "a.c", extra: "a.b", expected: <{"c":"b"}> but was: <{"b":"c"}> Different keys found in node "root", missing: "root.bar", extra: "root.foo", expected: <{"bar":"foo"}> but was: <{"foo":"bar"}> at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory.createException(ExceptionFactory.java:37) at net.javacrumbs.jsonunit.core.internal.ExceptionUtils.createException(ExceptionUtils.java:45) at net.javacrumbs.jsonunit.core.internal.Diff.failIfDifferent(Diff.java:613) at net.javacrumbs.jsonunit.assertj.JsonAssert.isEqualTo(JsonAssert.java:121) at demo.Demo.verifyJson(Demo.java:57) at demo.Demo.differentInNested(Demo.java:50) <1 internal line> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) ``` If the logging level is set to `DEBUG`, at the top we will also get this, but Intellij IDEA will not suggest to view the difference: ``` 23:16:39.184 [main] DEBUG net.javacrumbs.jsonunit.difference.diff -- JSON documents are different: Different keys found in node "", missing: "quux", extra: "foo", expected: <{"bat":{"baz":"bar"},"quux":{"qwe":"foo"}}> but was: <{"bat":{"quux":"qwe"},"foo":{"bar":"baz"}}> Different keys found in node "bat", missing: "bat.baz", extra: "bat.quux", expected: <{"baz":"bar"}> but was: <{"quux":"qwe"}> 23:16:39.187 [main] DEBUG net.javacrumbs.jsonunit.difference.values -- Comparing expected: {"quux":{"qwe":"foo"},"bat":{"baz":"bar"}} ------------ with actual: {"foo":{"bar":"baz"},"bat":{"quux":"qwe"}} ```

Output of unrelatedStructures.

``` org.opentest4j.AssertionFailedError: Different keys found in node "", missing: "quux", extra: "foo", Expected :{"bat":{"baz":"bar"},"quux":{"qwe":"foo"}} Actual :{"bat":{"quux":"qwe"},"foo":{"bar":"baz"}} at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError.getError(ExceptionFactory.java:55) <7 interal lines> at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError.(ExceptionFactory.java:46) at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory.createException(ExceptionFactory.java:37) at net.javacrumbs.jsonunit.core.internal.ExceptionUtils.createException(ExceptionUtils.java:45) at net.javacrumbs.jsonunit.core.internal.Diff.failIfDifferent(Diff.java:613) at net.javacrumbs.jsonunit.assertj.JsonAssert.isEqualTo(JsonAssert.java:121) at demo.Demo.verifyJson(Demo.java:57) at demo.Demo.unrelatedStructures(Demo.java:25) <31 interal lines> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) <9 interal lines> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) <28 interal lines> org.opentest4j.AssertionFailedError: Different keys found in node "bat", missing: "bat.baz", extra: "bat.quux", Expected :{"baz":"bar"} Actual :{"quux":"qwe"} at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError.getError(ExceptionFactory.java:55) <7 interal lines> at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError.(ExceptionFactory.java:46) at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory.createException(ExceptionFactory.java:37) at net.javacrumbs.jsonunit.core.internal.ExceptionUtils.createException(ExceptionUtils.java:45) at net.javacrumbs.jsonunit.core.internal.Diff.failIfDifferent(Diff.java:613) at net.javacrumbs.jsonunit.assertj.JsonAssert.isEqualTo(JsonAssert.java:121) at demo.Demo.verifyJson(Demo.java:57) at demo.Demo.unrelatedStructures(Demo.java:25) <31 interal lines> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) <9 interal lines> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) <28 interal lines> net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory$JsonAssertError: JSON documents are different: Different keys found in node "", missing: "quux", extra: "foo", expected: <{"bat":{"baz":"bar"},"quux":{"qwe":"foo"}}> but was: <{"bat":{"quux":"qwe"},"foo":{"bar":"baz"}}> Different keys found in node "bat", missing: "bat.baz", extra: "bat.quux", expected: <{"baz":"bar"}> but was: <{"quux":"qwe"}> at net.javacrumbs.jsonunit.core.internal.Opentest4jExceptionFactory.createException(ExceptionFactory.java:37) at net.javacrumbs.jsonunit.core.internal.ExceptionUtils.createException(ExceptionUtils.java:45) at net.javacrumbs.jsonunit.core.internal.Diff.failIfDifferent(Diff.java:613) at net.javacrumbs.jsonunit.assertj.JsonAssert.isEqualTo(JsonAssert.java:121) at demo.Demo.verifyJson(Demo.java:57) at demo.Demo.unrelatedStructures(Demo.java:25) <1 interal line> at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) at java.base/java.util.ArrayList.forEach(ArrayList.java:1511) ```

Let me know, if you now understand what I was talking about. I can probably try to explain with a complete test report as I think it'd be better.

---

Updated: Can confirm it's possible to do it on my side, catching MultipleFailuresError and extracting the necessary information from JsonUnit's AssertionFailedError. This probably isn't ideal though, as it'd practically mean I'd have to write a wrapper around JsonUnit...

[lukas-krecan]

Hi, I understand, but honestly, I do not know how to implement it.

1. In my opinion it's useful to show the difference as JsonUnit shows it today. Unfortunately, IntelliJ is able to show only one difference per assertion error so we would have to drop the current behavior in order to support your use-case.
2. The difference view in IntelliJ is based on string comparison. The reason why people are using JsonUnit and not plain string assert is that comparing JSONs is not the same as comparing Strings. In simple cases we could normalize the JSONs and then comare them as strings. But that would break for any nontrivial feature like "json-unit.ignore".