xcparse -s Illegal instruction: 4

[olevabel]

Hi,

When using the xcparse I am facing an issue. I navigate to the directory where .xcresult is at and call xcparse -s MY_PROJECT.xcresult ./screens, having previously created an empty screens dictionary in the root. The xcparse command is returning: Illegal instruction: 4

[selin194]

Hey @olevabel ! I execute the command xcparse -s output.xcresult ./screens I get some screenshots. How do you get .xcresult file? I execute the command here to get xcresult file: xcodebuild -workspace myApp.xcworkspace -scheme "UITests" -sdk iphonesimulator -destination 'platform=iOS Simulator,name=iPhone 8 Plus,OS=13.0' test -resultBundlePath output.xcresult Edit: I also created an empty folder called screens Thank you.

[abotkin-cpi]

Thanks for the report @olevabel! Could you check add the following information?

Xcode version: (Ex. "11.0 (11A420a)") Swift version: (Run "swift -version" from Terminal)

If you open up Console app and then run the command from Terminal, there may be a log from ReportCrash along the lines of "Saved crash report for xcparse" & a crash file path in DiagnosticReports. If so, please grab that crash report and either open it & paste the content or upload it.

If you're open to sharing the XCResult, you can attach it here or email me via my profile page. @rsukumar-cpi had a visualizer that could help us to see what's in your XCResult without having you attach sensitive data in a PR I've been meaning to review. I'll try and review that and see if we can get that added into xcparse for future debugging w/o the xcresult

[abotkin-cpi]

@olevabel One other thing, could you check that you're using an Xcode 11 XCResult and not an Xcode 10 XCResult? You can check this by finding the xcresult in Finder, right-clicking (CTRL+click) and choosing "Show Package Contents". If it's an Xcode 11 XCResult, you'll see a Data folder which will have a bunch of weird looking files. If it's an Xcode 10 XCResult, you'll see a folder structure with Attachments where your screenshots are.

Example:

Right now, xcparse assumes you're calling with Xcode 11 XCResults and not Xcode 10 & will cause the Illegal Instruction you mentioned when given an Xcode 10 XCResult. We should add in protection in xcparse to check the XCResult format prior to attempting parsing in the Xcode 11 style; we just didn't do it initially as we had a Bash script that extracted Xcode 10 attachments so it was a low priority to us.

[NikitaZamalyutdinov]

Hey, it's was reproduced for me too: I ran xcparse -s TestResults.xcresult screenshots/ and get error:

Fatal error: Error raised at top level: Swift.DecodingError.keyNotFound(ActionTestSummaryGroupCodingKeys(stringValue: "subtests", intValue: nil), Swift.DecodingError.Context(codingPath: [ActionTestPlanRunSummariesCodingKeys(stringValue: "summaries", intValue: nil), _JSONKey(stringValue: "Index 0", intValue: 0), ActionTestPlanRunSummaryCodingKeys(stringValue: "testableSummaries", intValue: nil), _JSONKey(stringValue: "Index 0", intValue: 0), ActionTestableSummaryCodingKeys(stringValue: "tests", intValue: nil), _JSONKey(stringValue: "Index 0", intValue: 0), ActionTestSummaryGroupCodingKeys(stringValue: "subtests", intValue: nil), _JSONKey(stringValue: "Index 1", intValue: 1)], debugDescription: "No value associated with key ActionTestSummaryGroupCodingKeys(stringValue: \"subtests\", intValue: nil) (\"subtests\").", underlyingError: nil)): file /BuildRoot/Library/Caches/com.apple.xbs/Sources/swiftlang/swiftlang-1001.8.63.13/swift/stdlib/public/core/ErrorType.swift, line 200
Illegal instruction: 4

Xcode version: "11.0 (11A420a)" Swift version: 5.1 (swiftlang-1100.0.270.13 clang-1100.0.33.7) Thank you.

[abotkin-cpi]

Thanks for the report @NikitaZamalyutdinov. Yeah, this seems like the strict parsing we're doing where the formatDescription says a value is non-optional & assuming that it means the key will be in the file is not the right assumption. It comes up much more with the Xcode 10 xcresults as the bare minimum structure seems to be added by xcresulttool to get going, but a bunch of non-optional keys are just not provided in the JSON output.

I'll try to get to this in the next day or two.

[maksim-sushkevich]

Hey there!

I have faced the issue below:

When I run xcparse -s TestResults2.xcresult ./screenshots I always get the error: error: unexpected argument case; use --help to list available arguments

The same error I get in interactive mode as well.

XCode Version 11.0 (11A420a) Apple Swift version 5.1 (swiftlang-1100.0.270.13 clang-1100.0.33.7)

Thank you!

[abotkin-cpi]

@maksim-sushkevich Thanks for the report! Hmm, that error seems to be coming from xcresulttool & sounds like we're not parsing your screenshots right (the argument case in this case isn't the arguments to xcparse, but what we're providing to "xcrun xcresulttool" in order to get out your screenshots).

Could you make a new issue report? I've got a suspicion on what it could be - I'm pretty sure our v0.1 release hard-coded some assumptions about the structure that doesn't hold true if you have simpler tests configurations. I'll likely have the changes for that later this week, but if it doesn't work, I'll likely need you to provide more debugging info from a new command we'll be adding in v0.2.

[abotkin-cpi]

@olevabel @NikitaZamalyutdinov @maksim-sushkevich Changes were integrated in 0.3 that should resolve the majority of these cases. Run brew upgrade xcparse to get that version. If you see this again, let me know!

[NikitaZamalyutdinov]

@abotkin-cpi

Unfortunately, still reproduced for me. I'm going to debug that issue a few days later.

[abotkin-cpi]

@NikitaZamalyutdinov If you can upload the crash log, I can also take a look. In Mojave and below, if you open a Finder window & click Go -> Go To Folder and input "~/Library/Logs/DiagnosticReports", you should be able to see your crash logs and see the xcparse one. In Catalina, you can get it by going to Console app and checking the side bar for Crash Reports.

Also double-check that brew successfully updated by running "brew info xcparse" and that you see a line about "/usr/local/Cellar/xcparse/0.3". One of our devs had Homebrew "upgrade" but it only upgraded the formula in the local repo due to a permission failure & didn't update xcparse until a second command for upgrade was issued. That would be easy to tell by seeing if "/usr/local/Cellar/xcparse" shows a different version than 0.3 in "brew info xcparse"

[NikitaZamalyutdinov]

@abotkin-cpi Thanks for answer, I've checked version of xcparse, actual is 0.3. My crash report: xcparse_2019-10-07-193321_admins-Mac-mini..crash.txt

[abotkin-cpi]

@NikitaZamalyutdinov Thanks for the crash report! That stack trace indicates something that should've been fixed with 0.3 by the addition of a Double specific method in XCResultDecoding.swift:264.

I do notice that xcparse in the crash report has a path that is not the usual Homebrew install location. Specifically, your crash report is saying xcparse is in "/usr/local/bin/xcparse" rather than "/usr/local/Cellar/xcparse/0.3/bin" (as Homebrew should add a sym link in /usr/local/bin to the true location over in /usr/local/Cellar). Is it possible when you were working on your feature request that you copied your local build of xcparse into /usr/local/bin in order to test your version from Terminal? If you do ls -la /usr/local/bin, you should see the following if you have the Homebrew version:

-rwxr-xr-x    1 abotkin  admin     555 Jul 16 20:27 xcodeproj
-rwxr-xr-x    1 abotkin  admin     525 Sep 17 17:18 xcov
lrwxr-xr-x    1 abotkin  admin      33 Oct  4 15:36 xcparse -> ../Cellar/xcparse/0.3/bin/xcparse
-rwxr-xr-x    1 abotkin  admin     549 Jul 16 20:27 xcpretty
-rwxr-xr-x    1 abotkin  admin     651 Jul 16 20:27 xcpretty-travis-formatter
lrwxr-xr-x    1 abotkin  admin      47 Oct  2 14:12 xliff2odf -> ../Cellar/translate-toolkit/2.4.0/bin/xliff2odf

If you're not seeing that it's a sym link to the location in Cellar, it's likely a local build was copied there at some point. You can do rm /usr/local/bin/xcparse and then do an uninstall & reinstall with Homebrew to correct it. Commands should be something like brew uninstall xcparse then brew install chargepoint/xcparse/xcparse

[kenji21]

reproduced this issue with current master 03b9ca9:

A test in https://github.com/openium/SwiftiumTestingKit just add an attachment, so run them with:

xcodebuild test -scheme STKTestApp -destination "name=iPhone 8 Plus" -resultBundlePath TestResults.xcresult

And then run:

~/github/xcparse/.build/debug/xcparse --screenshots TestResults.xcresult screenshots

which leads to this error:

Fatal error: Error raised at top level: Swift.DecodingError.keyNotFound(ActionTestSummaryGroupCodingKeys(stringValue: "subtests", intValue: nil), Swift.DecodingError.Context(codingPath: [ActionTestPlanRunSummariesCodingKeys(stringValue: "summaries", intValue: nil), _JSONKey(stringValue: "Index 0", intValue: 0), ActionTestPlanRunSummaryCodingKeys(stringValue: "testableSummaries", intValue: nil), _JSONKey(stringValue: "Index 0", intValue: 0), ActionTestableSummaryCodingKeys(stringValue: "tests", intValue: nil), _JSONKey(stringValue: "Index 0", intValue: 0), ActionTestSummaryGroupCodingKeys(stringValue: "subtests", intValue: nil), _JSONKey(stringValue: "Index 0", intValue: 0)], debugDescription: "No value associated with key ActionTestSummaryGroupCodingKeys(stringValue: \"subtests\", intValue: nil) (\"subtests\").", underlyingError: nil)): file /BuildRoot/Library/Caches/com.apple.xbs/Sources/swiftlang/swiftlang-1001.8.63.13/swift/stdlib/public/core/ErrorType.swift, line 200
Illegal instruction: 4

using Xcode 11.0

xcodebuild -version
Xcode 11.0
Build version 11A420a

[abotkin-cpi]

@kenji21 Thanks for the report! Yep, 0.3 wouldn’t cover the array keys being empty for non optional properties (0.3 only covers types like bool, double, string, int). Should have a build that corrects that in the next day. We should open a separate ticket for that.

EDIT: Create new ticket #16 & put explanation there for those curious.

[abotkin-cpi]

@kenji21 xcparse 0.3.1 is now out on Homebrew & should correct the issue you were seeing with the non-optional arrays. Upgrade using brew upgrade xcparse

[kenji21]

also works when running on multiple simulators via xcodebuild test -scheme STKTestApp -destination "name=iPhone 8 Plus" -destination "name=iPhone XS Max" -resultBundlePath TestResults.xcresult

[NikitaZamalyutdinov]

@abotkin-cpi Today, I've upgraded xcparse onto 0.3.1 and launch xcparse -s TestResults.xcresult/ screenshots/ , actual version doesn't crash, but doesn't extract any screenshot from 'xcresult'. I've opened 'xcresult' by double-clicking in Xcode and get access to all screenshots.

[abotkin-cpi]

@NikitaZamalyutdinov Interesting - seems we must be missing something in our parsing. Can you create a new issue & attach any information you can provide about the XCResult (do you use test run configurations yet or not, was this invoked with multiple run configurations, how many tests there are, etc.). I'll put up a branch that will allow for verbose logging that you could possibly use to help give more info without giving the full XCResult (and redact elements as necessary).

I'm aware of some issues parsing when having multiple run destinations & am actively investigating that, but I don't see the run destination icon in your screenshot, so I assume you're not running multiple devices in that test run.

[abotkin-cpi]

Think I've figured out the issue that's affecting @NikitaZamalyutdinov & @kenji21 with missing screenshots. Should have a fix posted tonight as part of #19

EDIT: @NikitaZamalyutdinov @kenji21 give it a try with 0.3.2

[kenji21]

tested with a project converted to xctestplan having two configurations (languages fr & en), runned on two simulators (XS Max, 8 Plus), and had all "4 screenshots" ✅