junit-team / junit5

✅ The 5th major version of the programmer-friendly testing framework for Java and the JVM
https://junit.org
Other
6.25k stars 1.45k forks source link

Add static factory methods that return `Supplier<String>` to format test errors #3841

Open shahryarkiani opened 1 month ago

shahryarkiani commented 1 month ago

I think it would be useful for there to be an easy way for users to get different output formats for their tests.

For example, in the case of asserting equality on a collection, each list is outputted horizontally on a single line. That can be hard to compare when the elements are of varying length.

For reference, an example output looks like this:

expected: <[short, a longer string, even longer string that is annoying]> but was: <[i am a small, i am not so small, tiny]>
Expected :[short, a longer string, even longer string that is annoying]
Actual   :[i am a small, i am not so small, tiny]

I think a good solution would be to expose some static factory methods that return a Supplier<String> that formats the output in a desired format, and the user can then pass it in. So a user could do something like: assertEquals(CollectionA, CollectionB, OutputFormat.ofVertical(CollectionA, CollectionB)); This wouldn't affect any existing user tests/code and would also be flexible to adding new formats. I think it's also not that easy for users to implement the functionality on their own, since getting the spacing right can be tough. A quick Supplier<String> I wrote up looks like this:

expected   |   actual
short | i am a small
a longer string | i am not so small
even longer string that is annoying | tiny

But if this seems like a good idea, I could write it so that we get something like this:

expected                            | actual
---------------------------------------------------
short                               | i am a small
a longer string                     | i am not so small
even longer string that is annoying | tiny

(Would also need to handle stuff like unequal collection sizes, maybe including indexes, etc.)

This output would appear above the existing default output, so it does duplicate some information.

I'm interested in implementing this myself and getting to contribute to JUnit.

mpkorstanje commented 1 month ago

Have you considered using an assertion library? The user guide recommends a few:

Even though the assertion facilities provided by JUnit Jupiter are sufficient for many testing scenarios, there are times when more power and additional functionality such as matchers are desired or required. In such cases, the JUnit team recommends the use of third-party assertion libraries such as AssertJ, Hamcrest, Truth, etc. Developers are therefore free to use the assertion library of their choice.

Depending on the size of the list and the size of its content, I know that AssertJ already switches between a horizontal and vertical layout. It also sounds look like you may be interested in the Overriding error messages feature from AssertJ.