allankp / pytest-testrail

pytest plugin for integration with TestRail, for creating testruns and updating results
MIT License
95 stars 125 forks source link

Issue-89: Support xdist plugin to run test in parallel #90

Closed YakovLittle closed 5 years ago

YakovLittle commented 5 years ago

Related to https://github.com/allankp/pytest-testrail/issues/89 Now we have Test Run for each xdist worker, but skipping to publish Untested test cases

P.S. Added logging for using pytest logger (e.g. pytest.ini) ` [pytest] log_cli = 1 log_cli_level = INFO log_cli_format = %(message)s

log_file = log/pytest.log log_file_level = DEBUG log_file_format = %(asctime)s.%(msecs)03d {%(filename)s::%(funcName)s::%(lineno)d} [%(levelname)s] %(message)s log_file_date_format=%Y-%m-%d %H:%M:%S `

YakovLittle commented 5 years ago

To collect all results in one Test Run -> I've started to use junit parser to import results from xml report file after all xdist workers

danchownow commented 5 years ago

To collect all results in one Test Run -> I've started to use junit parser to import results from xml report file after all xdist workers

Can you expand on how you were able to implement this? Having the same issue.

YakovLittle commented 5 years ago
  1. Modify your base conftest.py to collect testrail_id, e.g. ` def pytest_collection_modifyitems(session, config, items): tags = ["jira_id", "testrail_id"]

    for item in items: for tag in tags: for marker in item.iter_markers(name=tag): value = marker.args[0] item.user_properties.append((marker.name, value))`

  2. Using junitparser to collect Test cases, e.g. `from junitparser import JUnitXml, Element, Attr

class PropertiesElement(Element): _tag = "properties"

class PropertyElement(Element): _tag = "property" name = Attr() value = Attr()

def collect_test_cases(ctx, report): """ Collect test cases with TestRail ID :param ctx: CLI Context object :param report: path to report file :return (list): """ test_cases = [] try: xml = JUnitXml.fromfile(report) except Exception as exc: # pylint: disable=broad-except ctx.vlog(exc) exit(1)

for case in xml:
    case.append(PropertiesElement())
    for case_properties in case.iterchildren(PropertiesElement):
        case_properties.append(PropertyElement())
        for case_property in case_properties.iterchildren(PropertyElement):
            if case_property.name != "testrail_id":
                continue
            comment = ""
            if not case.result:
                status = 1  # passed
            elif "Skipped" in str(type(case.result)):
                status = 6
                comment = case.result.message[-4000:]
            elif "Failure" in str(type(case.result)):
                status = 5
                comment = case.result.message[-4000:]
            else:
                status = 3  # untested
            duration = case.time
            if duration:
                duration = 1 if duration < 1 else int(round(duration))
                duration = str(duration) + "s"
            data = {
                "case_id": case_property.value[1:],
                "status_id": status,
                "comment": comment,
                "elapsed": duration
            }
            test_cases.append(data)

return test_cases

`

  1. Create Test run
  2. Update Test run
  3. Close Test run