wolever / parameterized

Parameterized testing with any Python test framework
Other
836 stars 105 forks source link

feature: report line number of the data when data is declared with param() #105

Open mathieucaroff opened 4 years ago

mathieucaroff commented 4 years ago

When the user uses param() to declare their data, it is possible to know the line number and location of the place where param was called. This information could be memorized by parameterized, and later used to indicate which piece of data caused the failure of the test.

Example

test_number.py:

from unittest import TestCase

from parameterized import parameterized, param

class AnimalTestCase(TestCase):
    @parameterized.expand(
        [  # line 8
            param("inferior", left=1, right=2, inferior=True),  # line 9
            param("equal", left=2, right=2, inferior=True),  # line 10
            param("superior", left=3, right=2, inferior=False),  # line 11
        ]
    )
    def test_comparison(self, _, left, right, inferior):
        self.assertEqual(left < right, inferior)

Currently:

$ python -m unittest test_number.py
.F.
======================================================================
FAIL: test_comparison_1_equal (test_number.NumberTestCase)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/mathieucaroff/venv/lib/python3.6/site-packages/parameterized/parameterized.py", line 530, in standalone_func
    return func(*(a + p.args), **p.kwargs)
  File "/home/mathieucaroff/project/test_number.py", line 15, in test_comparison
    self.assertEqual(left < right, inferior)
AssertionError: False != True

----------------------------------------------------------------------
Ran 3 tests in 0.000s

FAILED (failures=1)

There are a number of ways to do it. Here are a few proposals, from the one I like the most to the one I like the least. I'm not sure that all of them can work, but the simplest ones necessarily work.

adding a line in the stack trace
  File "/home/mathieucaroff/project/test_number.py", line 15, in test_comparison
    self.assertEqual(left < right, inferior)
  File "/home/mathieucaroff/project/test_number.py", line 10
    param("equal", left=2, right=2, inferior=True),  # line 10
AssertionError: False != True
adding the information to the error
  File "/home/mathieucaroff/project/test_number.py", line 15, in test_comparison
    self.assertEqual(left < right, inferior)
AssertionError: False != True, with data from: "/home/mathieucaroff/project/test_number.py", line 10
adding the information to the name of the generated function
======================================================================
FAIL: test_comparison_1_equal__line_10 (test_number.NumberTestCase)
----------------------------------------------------------------------