UnitTest: How to display assertion failures for iterables?

[ben-albrecht]

When an assertion fails for an iterable, such as arrays, lists, tuples, etc., what information should be included in the error message?

The error message information could include some combination of the following:

• The indices that failed the assertion
• The values that failed the assertion
• All values of the iterable

As of writing this issue, the UnitTest module special-cases 1D iterables and prints all of this information:

AssertionError: Arrays differ: '[1, 5, 7]' != '[1, 7, 6]'
First differing element at index 2:
'5'
'7'

For higher-order arrays, the assertion failure includes the values of all elements:

AssertionError: assert failed - 
0 0 1
0 0 0
0 0 0
!=
0 0 0
0 0 0
1 0 0

I think we should choose an error message strategy that:

• scales well with number of elements
• scales well with number of differing elements
• is consistent across dimensions

Note that scaling here refers to the usefulness of the output as the array sizes increase, e.g. outputting the elements of 1Mx1M array is not scalable.

[ben-albrecht]

One potential strategy is to print all information for all dimensions of arrays/iterables and to utilize ellipses to make the output more concise for some arbitrary threshold, e.g.

# Small array
AssertionError: Arrays differ: 
1 5 7
!=
1 7 6
Differing elements at 1 index:
2: 5 != 7

# Large array
AssertionError: Arrays differ: 
1 5 ... 7 2
!=
1 5 ... 7 2
Differing elements at 12 indices:
4: 1 != 3
8: 2 != 4
...
82: 12 != 9
99: 5 != 7