Open JeffreyDevloo opened 7 years ago
@JeffreyDevloo @pploegaert Can you provide some pointers for me so i can see how this might be done?
for test in tests:
mod = importlib.import_module('{0}.main'.format(test))
module_result = mod.run(blocked)
if hasattr(TestrailResult, module_result['status']): # check if a test has failed, if it has failed check if we should block all other tests
if getattr(TestrailResult, module_result['status']) == TestrailResult.FAILED and fail_on_failed_scenario:
if 'blocking' not in module_result:
blocked = True # if a test reports failed but blocked is not present = by default blocked == True
elif module_result['blocking'] is not False:
blocked = True # if a test reports failed but blocked != False
else:
raise AttributeError("Attribute `{0}` does not exists as status in TestrailResult".format(module_result['status']))
# add test to results & also remove possible EXCLUDE_FLAGS on test name
results[test.replace(AutoTests.EXCLUDE_FLAG, '')] = module_result
logger.info("Finished tests.")
if send_to_testrail:
logger.info("Start pushing tests to testrail.")
plan_url = AutoTests.push_to_testrail(results, only_add_given_cases=only_add_given_results)
return results, plan_url
return results, None
Place the sending within the for loop is what I suggest
Problem description
There is a currently a flaw with the autotests: they push all their results at once. On failure, it currently stores all log files between the time of test in memory. Worst case this can mean 2.5 hours of logging.
The pushing should be done after every test, rather than them all.