fundakol / pytest-jira-xray

Plugin for pytest to upload test results to JIRA XRAY
Apache License 2.0
27 stars 18 forks source link

Filter tests when testplan option is provided #113

Open sanderploegsma opened 5 months ago

sanderploegsma commented 5 months ago

When working with a large collection of tests and multiple test plans, it would make sense if this plugin was able to filter the tests based on whether they are part of the provided Xray test plan.

I can think of two ways of supporting this:

  1. Query the Xray API to get a list of test ids for a given test plan
  2. Add a pytest marker to each test with the test plans that contain it

The first option is more versatile as it doesn't require keeping the test decorators in sync with Xray, but it also adds some complexity to handle cases when querying the API fails.

fundakol commented 5 months ago

My intention writing this plugin wasn't to implement Rest API for Jira/Xray except the one for uploading results. Also I don't use Xray anymore so I am not able to test this plugin with new features. But if you want to implement this feature and test it with your Jira server that's fine for me.

Probably this may look like that:

def pytest_collection_modifyitems(config, items):
    if not (test_plan_id := config.option.testplan):
        return
    selected_items = []
    deselected_items = []
    selected_tests: set[str] = get_tests_for_testplan(test_plan_id)
    for item in items:
        if marker := item.get_closest_marker('xray'):
            if set(marker.args).issubset(test_plan_id):
                selected_items.append(item)
            else:                
                deselected_items.append(item)                                
        else:
            selected_items.append(item)  # run other tests without xray marker
    if deselected_items:
        config.hook.pytest_deselected(items=deselected_items)
    items[:] = selected_items