search

pchomik / pytest-spec

Library pytest-spec is a pytest plugin to display test execution output like a SPECIFICATION.
GNU General Public License v2.0
100 stars 19 forks source link

Strange rendering of parameterized tests #40

Closed paxcodes closed 3 years ago

paxcodes commented 3 years ago

Problem Parameterized tests with tuples should output tuples.

To Reproduce

RSS_URLS = {"https://pypi.org/rss/project/pytest-spec/releases.xml": 7}

@fixture(scope="module")
def actualItems(request):
    return request.param

@fixture(scope="module")
def actualItemsByDate(request):
    return request.param

@mark.parametrize(
    "actualItems", RSS_URLS.keys(), indirect=True,
)
def test_items_are_wonderful(actualItems):
    pass

@mark.parametrize(
    "actualItemsByDate", RSS_URLS.items(), indirect=True,
)
def test_it_can_retrieve_items_published_within_a_date_range(actualItemsByDate):
    pass

Actual Output

  ✓ Items are wonderful[https://pypi.org/rss/project/pytest-spec/releases
  ✓ It can retrieve items published within a date range[actualItemsByDate0]

Expected Output

  ✓ Items are wonderful[https://pypi.org/rss/project/pytest-spec/releases]
  ✓ It can retrieve items published within a date range[https://pypi.org/rss/project/pytest-spec/releases, 7]
paxcodes commented 3 years ago

Another output I didn't expect:

Given a parametrized test,

class Test_Something:

  @mark.parametrize(
        "givenDateTime, expectedStartDate, expectedEndDate",
        [
            (
                "2021-03-19 03:00:01",
                datetime(2021, 3, 12, 3, 0, 0, 0),
                datetime(2021, 3, 19, 3, 0, 0, 0),
            ),
        ]
  )
  def test_me(self, givenDateTime: str, expectedStartDate: datetime, expectedEndDate: datetime): ...

Actual Output

Something:
  ✓ Me

Expected Output

Something:
  ✓ Me ["2021-03-19 03:00:01", datetime(2021, 3, 12, 3, 0, 0, 0), datetime(2021, 3, 19, 3, 0, 0, 0)]
pchomik commented 3 years ago

Hello,

Thank you @paxcodes for reporting issue. I was able to reproduce your problem, Let's investigate them one by one.

Regarding first test problem the issue is not related to pytest-spec plugin but pytest itself. To see that please execute two commands:

The second command will show you how pytest is tagging your parametrize cases. Output in pytest and pytest-spec are the same. I cannot change this because I'm getting such information from pytest.

Regarding second case I have a problem with your example because I have this output:

  ✓ Me[2021-03-19 03:00:01-expectedStartDate0-expectedEndDate0]

The name is strange but such name I received from pytest

Please discuss parametrize issue with pytest developers. Good luck.