MobileNativeFoundation / XCLogParser

Tool to parse Xcode and xcodebuild logs stored in the xcactivitylog format
Apache License 2.0
1.75k stars 125 forks source link

Invalid xcactivitylog file #139

Open chatwyn opened 3 years ago

chatwyn commented 3 years ago

Hi, When I use Xcode to build, It will generate xcactivitylog. But When I use xcodebuild cli, For example

xcodebuild -workspace xx.xcworkspace -scheme xx -sdk iphoneos -showBuildTimingSummary -resultBundlePath ./a/a -derivedDataPath ./a clean build

it will generate two xcactivitylog, One is a clean job log, and one is a build job log. But the build job log is not a valid gzip. I have to wait one day, but nothing changes.

So, Is it possible to crash in generate xcactivitylog?

ptrkstr commented 3 years ago

Similar to myself however for me it depends on the size of the project. If I create a new xcode project, xcodebuild works. If I use it on an existing large xcode project, I receive .xcactivitylog is not a valid xcactivitylog file. Even after waiting an hour and running xclogparser, I get the same error. This is my command:

xcodebuild \
-workspace XXX.xcworkspace \
-scheme XXX \
-destination generic/platform=iOS \
-resultBundlePath ~/Desktop/BuildTimeReports/BuildLogs/Result.xcresult \
clean build-for-testing \
OTHER_SWIFT_FLAGS="-Xfrontend -debug-time-function-bodies -Xfrontend -debug-time-expression-type-checking"

When I run gzip -t XXX.xcactivitylog, it fails to pass validation. I believe this is a problem with xcodebuild, not with xclogparser.

chatwyn commented 3 years ago

Yes, I found if I use hmap replace header search path, it can generate .xcactivitylog correctly. So I think it may be an argument to long error in xcodebuild. But I don't know how to create issue to xcodebuild.

ptrkstr commented 3 years ago

Yes, I found if I use hmap replace header search path, it can generate .xcactivitylog correctly. So I think it may be an argument to long error in xcodebuild. But I don't know how to create issue to xcodebuild.

Thanks for responding, could you give a bit more information on what "hmap replace header search path" is?

ptrkstr commented 3 years ago

I've switched to using Xcode directly to build instead of xcodebuild and that works. I do need to wait a few seconds after the build is finished before running xclogparser.

chatwyn commented 3 years ago

https://tech.meituan.com/2021/02/25/cocoapods-hmap-prebuilt.html

chatwyn commented 3 years ago

me too. I think is xcodebuild bug, but I don't know how to submit bug to xcodebuid

anton-plebanovich commented 3 years ago

It looks like xcodebuild bug. It finishes execution before .xcactivitylog file is fully created. I was able to work around this behavior by using clean build clean instead of clean build so one additional clean is happening after build and buy us some time.

anton-plebanovich commented 3 years ago

The other thing is that xcodebuild produces colored output that can't be properly parsed because of strings like:

"\u{1B}[1mCompileSwift\u{1B}[0m normal x86_64 /Users...

getDetailType(signature:) fails:

        case Prefix("CompileSwift "):
            return .swiftCompilation

I was able to work around using COLOR_DIAGNOSTICS=NO CLANG_COLOR_DIAGNOSTICS=NO options for `xcodebuild.

stevelandeyasana commented 2 years ago

I had this problem as well, and I "fixed" it by adding sleep 2 to the script that runs xclogparser.

imWildCat commented 2 years ago

I'm seeing the same problem as well using Xcode Version 13.2.1 (13C100).

xcodebuild with incremental builds is fine. But clean builds didn't work at all.

vmzhivetyev commented 2 years ago

any updates?

Sherlouk commented 1 year ago

Just adding as a user of XCMetrics (which relies on this project), I too am running into problems where the underlying xcactivitylog is (for some reason) invalid on a medium sized project. Works fine via Xcode, but problematic via xcodebuild.

Open to any workarounds or fixes to this. Xcode 14.1.

BalestraPatrick commented 1 year ago

As written above, this seems to be because Xcode closes the file handler or similar way too early without letting the file writer flush the contents of the file completely to disk. This should be a bug report to Xcode, and you can workaround it as described above by adding a sleep or running another action (such as a clean action) after your build.

Sherlouk commented 1 year ago

such as a clean action

Feels extremely janky, but this has worked for us. Though this exasperate an issue on XCMetrics where clean builds are pushing up their logs.

malcommac commented 1 year ago

We've faced the same issue by using fastlane's gym command. In our case by disabling analyze_build_time option (= false) generated xcactivitylog file seems correct.