getslash / slash

The Slash testing infrastructure
https://getslash.github.io/slash
Other
75 stars 38 forks source link

result_summary isn't called although it should #1063

Open hefux opened 1 year ago

hefux commented 1 year ago

According to the documentation, result_summary is

Called at the end of the execution, when printing results

But it appears that this doesn't work as expected, a minimal reproducible example:

import slash

def test_passes():
    assert 2 == 1 + 1

def test_fails():
    assert 2 == 1 + 2

def list_tests(prefix):
    print(prefix)
    for result in slash.context.session.results.iter_test_results():
        print(result.test_metadata.address)

@slash.hooks.result_summary.register
def list_result_summary():
    list_tests("result_summary")

@slash.hooks.session_end.register
def list_session_end():
    list_tests("session_end")

I would expect to have 2 calls to list_tests, one with the "result_summary" prefix and another one with the "session_end" prefix, but only the "session_end" one is printed.

In this code: https://github.com/getslash/slash/blob/3f8ca7228dd88afa22619a7ffa88fe9a574347ca/slash/app.py#L159-L160 result_summary is called only when exc_type is not None, I am not sure that this condition makes sense.