Updated error messages for new(?) FB Errors format

[PaulTaykalo]

In some cases (new version may be FBSnapshot Library will generate errors in slighlty different format)

[PaulTaykalo]

@Antondomashnev Idk what is the reason. It can be that swift/objc interoperability issues, or Xcode shows errors differently for different XCTestCase runs. Can't say. I don't see lines with (no error) in the log. So I thought that this can be some kind of snapshot library change. But it also can be that XCTest changed its output.

[Antondomashnev]

Hm, what Xcode version do you use and what ios-snapshot version do you use?

[PaulTaykalo]

So. Xcode 8.3.3 Deployment Target 9.0 SDK: 10.3 XCTests framework

 pod 'FBSnapshotTestCase', '~> 2.1.4'

Output

Test Case '-[MyProjectSnapshotTests.MyProjectSnapshotTests testAddTextRotated]' started.
2017-09-20 21:55:25.865 Project[36269:9222195] If you have Kaleidoscope installed you can run this command to see an image diff:
ksdiff "/Users/me/Projects/myProject/ios-app/MyProjectSnapshotTests/FailureDiffs/MyProjectSnapshotTests.MyProjectSnapshotTests/reference_testAddTextRotated@2x.png" "/Users/me/Projects/myProject/ios-app/MyProjectSnapshotTests/FailureDiffs/MyProjectSnapshotTests.MyProjectSnapshotTests/failed_testAddTextRotated@2x.png"
/Users/me/Projects/myProject/ios-app/MyProjectSnapshotTests/MyProjectSnapshotTests.swift:58: error: -[MyProjectSnapshotTests.MyProjectSnapshotTests testAddTextRotated] : failed - Snapshot comparison failed: Optional(Error Domain=FBSnapshotTestControllerErrorDomain Code=4 "Images different" UserInfo={NSLocalizedFailureReason=image pixels differed by more than 0.00% from the reference image, FBDiffedImageKey=<UIImage: 0x618000282710> size {400, 400} orientation 0 scale 2.000000, FBReferenceImageKey=<UIImage: 0x600000c90db0> size {400, 400} orientation 0 scale 2.000000, FBCapturedImageKey=<UIImage: 0x618000282940> size {400, 400} orientation 0 scale 2.000000, NSLocalizedDescription=Images different})
/Users/me/Projects/myProject/ios-app/MyProjectSnapshotTests/MyProjectSnapshotTests.swift:58: error: -[MyProjectSnapshotTests.MyProjectSnapshotTests testAddTextRotated] : failed - Snapshot comparison failed: Optional(Error Domain=FBSnapshotTestControllerErrorDomain Code=1 "Unable to load reference image." UserInfo={NSLocalizedFailureReason=Reference image not found. You need to run the test in record mode, NSLocalizedDescription=Unable to load reference image., FBReferenceImageFilePathKey=/Users/me/Projects/myProject/ios-app/MyProjectSnapshotTests/ReferenceImages_32/MyProjectSnapshotTests.MyProjectSnapshotTests/testAddTextRotated@2x.png})

As far as I see - it's not FBSnapshot output that changed - it's more a format of the XCTest Framework?

[Antondomashnev]

Everything seems as expected. The FBSnapshotsViewer looks for If you have Kaleidoscope installed you can run this command to see an image diff and use that line to parse information. Another two lines are not parsed. Let me know if there is anything else I can help with 😄

[PaulTaykalo]

@Antondomashnev Please take a look at the PR itself. Without this PR snapshot viewer cannot determine that there was an error. Currently, it expects that the error string will have format

 : ((noErrors) is true) failed - Snapshot comparison failed

And if you take a look at errors strings that were provided - there's no that substring. So in my case, FBSnapshotViewer won't give an alert since it won't find error message in the desired format

[PaulTaykalo]

@Antondomashnev Just re-read your previous comment. Will check then and give you an update. Anyway, in my case it was just ignoring errors 🤷‍♂️

[Antondomashnev]

@PaulTaykalo hey, have you checked the errors? 😄

[PaulTaykalo]

Yeah. Kind'a. Too many assertions - hard to debug :) Doesn't have too much time right now. but still. mine works. that version that in master - cannot find the diffs from the log. Will investigate more a bit later this week.

[Antondomashnev]

@PaulTaykalo thanks and weird things. Do you mind to provide the log file that isn't parsed correctly?

[PaulTaykalo]

@Antondomashnev Ookay. here's how you can test that current version is not working From what I see it won't work for Swift-file-test-cases 1) Setup Demo project of FBSnapshotTestCase pod try FBSnapshotTestCase

2) Open the project and set recording mode in both files (Swift/ ObjC) to true 3) set recording mode in both files (Swift/ ObjC) to false 4) Change something in Swift file to expect failure in Snapshot 5) See that SnapshotViewer is not able to find the files

[Antondomashnev]

@PaulTaykalo thanks a log I'll dig into this on the weekend.

[Antondomashnev]

Close in favor of #64