Provide an extension point and types hierarchy for rich assertion failure information

[nskvortsov]

Some test frameworks or test environments can provide more structured information about the failure than a pair of strings (expected/actual). For example, if expected and actual values are stored in text files on disk, then the paths to those files can be included.

Suggested entry point: getAdditionalInfo() method in AssertionFailedError

Types hierarchy can be similar to JUnit's TestSource. A serializable marker interface AssertionFailedInfo extends Serializable A set of implementations that can be type-checked in runtime. E.g.

FileComparisonInfo implements AssertionFailedInfo, FileComparisonData

This can also cover some other requests, e.g. #72

[marcphilipp]

@nskvortsov Thanks for raising the issue! Would FileComparisonInfo and/or FileComparisonData be part of OpenTest4J as well?

[nskvortsov]

I think yes, they should be. At least, it will allow other test frameworks to implement it. I remember there was a discussion about interfaces vs. implementations in the OTA4J here. It states that one can not catch an interface, thus the AssertionFailedError is a concrete class. For additional information, that one does not need to catch an interface should be sufficient.

[marcphilipp]

@nskvortsov Is this related to #72? /cc @vlsi

[nskvortsov]

Yes, it is related. This rich assertion failure infomation can cover the use-case described in #72 I prefer the API-based approach over the json-based text protocol.

[vlsi]

Java API-based approach can't work with, say, Python or BASH tests though