search

reportportal / agent-python-pytest

Framework integration with PyTest
Apache License 2.0
95 stars 102 forks source link

ReportPortal shows test run as "To Investigate" when pytest filtering result in no tests running #298

Closed davebrochu-es closed 2 years ago

davebrochu-es commented 2 years ago

name: Bug report about: Create a report to help us improve title: ' labels: bug assignees: ''

Describe the bug Sometimes pytest cases are written such that they only run on specific configurations, versions, or types of systems (functional testing). When pytest launches and collects tests to run, it is possible and acceptable that the collection may result in 0 tests to run. When this happens while using the pytest-reportportal plugin, a test result is added to ReportPortal with a result of "To Investigate". Rather than report a test result that never happened, there should be no test result reported for runs yielding 0 test executions.

Steps to Reproduce Steps to reproduce the behavior:

  1. Run pytest with the pytest-reportportal plugin and provide a filter that should result in 0 tests being selected. Ex "-k there_are_no_tests_by_this_name"

Expected behavior There aren't any results reported to ReportPortal when pytest finds 0 tests to execute during the collection phase.

Actual behavior A test result displays as "To Investigate"

Package versions 5.6.0, pytest-reportportal = 5.0.11

Additional context The root cause of this issue appears to be that the finish_test call send a result of None. Instead, if 0 tests are run it may be acceiptable to mark the run as Passed.

ReportPortal - Finish launch: request_body={'end_time': '1643993315923', 'status': None}

HardNorth commented 2 years ago

It doesn't make sense making empty runs. In most other cases it means problem. So no, I don't see any reasons to change behaviour. For your particular case you can modify the agent to have them empty.