search

theappbusiness / TABTestKit

Library designed to make writing and maintaining automated tests for iOS applications. This includes automation of bio-metrics and controlling of mock servers
MIT License
59 stars 13 forks source link

Add failed step and scenario information to failure messages #80

Closed KaneCheshire closed 4 years ago

KaneCheshire commented 4 years ago

When steps fail it can be difficult to understand exactly where in the test function things failed, especially on CI.

TTK already tries to help with this by adding a text attachment with lots of info when a failure occurs, but the log in the CI output can still be really unhelpful and just say something like "assert true failed".

I think we can improve this, since the most recently executed Scenario and Step are stored, including the line and file. We can add custom failure messages to XCTAssertTrue etc, and also potentially pass in the file and line from the most recent step (if there is one).

zacoid55 commented 4 years ago

This would be an amazing next enhancement. Is this utilising the file: file, line: line feature?

KaneCheshire commented 4 years ago

Yeppers! I think it's even easier to solve than I thought, since we already intercept the recordFailure call in TABTestCase, we can just check the latest step from Step.current and then change the file / line that is recorded when calling super.recordFailure

zacoid55 commented 4 years ago

Oh nice, are you currently working on this or is this something you'd not be able to get to for a while? Just so I know not to step on toes!

KaneCheshire commented 4 years ago

Already got a POC working so I'll try and get it into PR asap :)

KaneCheshire commented 4 years ago
image
zacoid55 commented 4 years ago

Oh amazing 😍