Uploading dSYM to Crashlytics hangs indefinitely in GitHub Actions environment

vadimbelyaev commented 2 years ago

Description

I have an open source iOS app that uses Firebase SDK 9.3.0 and I use GitHub Actions as a CI.

I recently attempted to upgrade to 9.5.0 and found out that my GitHub Actions builds started hanging on the last build phase, which uploads dSYM to Crashlytics. In one of my nightly builds it'd been hanging for about 5 hours until the runner was terminated by timeout.

I reverted back to 9.3.0 and it all started working fine again.

I then attempted to upgrade to 9.4.1 with the same result: the upload again started to hang indefinitely. The reproducibility rate is 100%.

However, it only impacts the GitHub Actions environment: when I build locally on my computer, it all works as expected with both 9.4.1 and 9.5.0.

The dSYM uploading build step in my project is configured exactly according to the official documentation: Get started with Firebase Crashlytics.

So I'm wondering: has anything changed in 9.4.1 regarding the dSYM uploads that might cause this issue in the GitHub Actions environment?

Reproducing the issue

Take an iOS app that uses Crashlytics with a build step that uploads dSYM to Crashlytcis
Upgrade Firebase SDK to 9.4.1 or 9.5.0
Run a CI build in GitHub Actions

If it helps, here's the repository in question: https://github.com/vadimbelyaev/TbilisiCleanups. My commits to the main branch from d779a4ba19ca3dcd868ddc949535cbca87a6bdb7 to f600f742093df3a653f2edc9faf2ef812262ed2f are attempts to narrow the issue down.

Firebase SDK Version

9.4.1

Xcode Version

13.4.1

Installation Method

Swift Package Manager

Firebase Product(s)

Analytics, Authentication, Crashlytics, Firestore, Messaging

Targeted Platforms

iOS

Relevant Log Output

Thu, 01 Sep 2022 22:26:21 GMT
[22:26:21]: ▸ Copying GoogleService-Info.plist
5820
Thu, 01 Sep 2022 22:27:05 GMT
[22:27:05]: ▸ Processing Info.plist
5821
Thu, 01 Sep 2022 22:27:05 GMT
[22:27:05]: ▸ Running script 'SwiftLint'
5822
Thu, 01 Sep 2022 22:27:05 GMT
[22:27:05]: ▸ Generating 'TbilisiCleanups.app.dSYM'
5823
Thu, 01 Sep 2022 22:27:25 GMT
[22:27:25]: ▸ Running script 'Upload dSYM to Crashlytics'
5824
Fri, 02 Sep 2022 03:17:55 GMT
Error: The operation was canceled.

If using Swift Package Manager, the project's Package.resolved

Expand Package.resolved snippet

```json { "pins" : [ { "identity" : "abseil-cpp-swiftpm", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/abseil-cpp-SwiftPM.git", "state" : { "revision" : "d302de612e3d57c6f4afaf087da18fba8eac72a7", "version" : "0.20220203.1" } }, { "identity" : "appauth-ios", "kind" : "remoteSourceControl", "location" : "https://github.com/openid/AppAuth-iOS.git", "state" : { "revision" : "33660c271c961f8ce1084cc13f2ea8195e864f7d", "version" : "1.5.0" } }, { "identity" : "async-http-client", "kind" : "remoteSourceControl", "location" : "https://github.com/swift-server/async-http-client.git", "state" : { "revision" : "794dc9d42720af97cedd395e8cd2add9173ffd9a", "version" : "1.11.1" } }, { "identity" : "boringssl-swiftpm", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/boringssl-SwiftPM.git", "state" : { "revision" : "79db6516894a932d0ddaff3b05b9da1e4f6c4069", "version" : "0.9.0" } }, { "identity" : "facebook-ios-sdk", "kind" : "remoteSourceControl", "location" : "https://github.com/facebook/facebook-ios-sdk.git", "state" : { "revision" : "e14f349945ba3f9516e3292107f42be19fe9fbe5", "version" : "12.3.2" } }, { "identity" : "firebase-ios-sdk", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/firebase-ios-sdk.git", "state" : { "revision" : "cbaa7ce7063d5ab22414962ec63366ccbba26034", "version" : "9.4.1" } }, { "identity" : "firebaseui-ios", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/FirebaseUI-iOS.git", "state" : { "revision" : "6c2993a144ac768ce9118c7feb3416960c36f194", "version" : "12.2.0" } }, { "identity" : "googleappmeasurement", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleAppMeasurement.git", "state" : { "revision" : "f54f60d0164d887e1174fa51ab2efe48a8e9d178", "version" : "9.3.0" } }, { "identity" : "googledatatransport", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleDataTransport.git", "state" : { "revision" : "b905c49326b72211531ed9d7baa02d724828a8dc", "version" : "9.1.4" } }, { "identity" : "googlesignin-ios", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleSignIn-iOS", "state" : { "revision" : "5ce850448e89500aca5b095af7247eb46dc0ca18", "version" : "6.2.3" } }, { "identity" : "googleutilities", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GoogleUtilities.git", "state" : { "revision" : "f4abe56ce62a779e64b525eb133c8fc2a84bbc1f", "version" : "7.7.1" } }, { "identity" : "grpc-ios", "kind" : "remoteSourceControl", "location" : "https://github.com/grpc/grpc-ios.git", "state" : { "revision" : "2af4f6e9c2b18beae228f50b1198c641be859d2b", "version" : "1.44.2-grpc" } }, { "identity" : "gtm-session-fetcher", "kind" : "remoteSourceControl", "location" : "https://github.com/google/gtm-session-fetcher.git", "state" : { "revision" : "4e9bbf2808b8fee444e84a48f5f3c12641987d3e", "version" : "1.7.2" } }, { "identity" : "gtmappauth", "kind" : "remoteSourceControl", "location" : "https://github.com/google/GTMAppAuth.git", "state" : { "revision" : "b9d1683be336ba8c8d1c6867bafeb056a5399700", "version" : "1.3.0" } }, { "identity" : "jmespath.swift", "kind" : "remoteSourceControl", "location" : "https://github.com/adam-fowler/jmespath.swift.git", "state" : { "revision" : "4513d319c4aaa6c3b2ac18e1e6566a803515ad91", "version" : "1.0.2" } }, { "identity" : "leveldb", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/leveldb.git", "state" : { "revision" : "0706abcc6b0bd9cedfbb015ba840e4a780b5159b", "version" : "1.22.2" } }, { "identity" : "nanopb", "kind" : "remoteSourceControl", "location" : "https://github.com/firebase/nanopb.git", "state" : { "revision" : "819d0a2173aff699fb8c364b6fb906f7cdb1a692", "version" : "2.30909.0" } }, { "identity" : "nuke", "kind" : "remoteSourceControl", "location" : "https://github.com/kean/Nuke", "state" : { "revision" : "ed09ce3d28273804d127cdac42841a52f568076d", "version" : "11.1.1" } }, { "identity" : "promises", "kind" : "remoteSourceControl", "location" : "https://github.com/google/promises.git", "state" : { "revision" : "3e4e743631e86c8c70dbc6efdc7beaa6e90fd3bb", "version" : "2.1.1" } }, { "identity" : "sdwebimage", "kind" : "remoteSourceControl", "location" : "https://github.com/SDWebImage/SDWebImage.git", "state" : { "revision" : "3e48cb68d8e668d146dc59c73fb98cb628616236", "version" : "5.13.2" } }, { "identity" : "soto", "kind" : "remoteSourceControl", "location" : "https://github.com/soto-project/soto.git", "state" : { "revision" : "9c938aadbbb33d6ed54d04dd6ba494f7f12e0905", "version" : "6.0.0" } }, { "identity" : "soto-core", "kind" : "remoteSourceControl", "location" : "https://github.com/soto-project/soto-core.git", "state" : { "revision" : "8e63c0f80db61f01c346f5109863bc2be29093e7", "version" : "6.0.0" } }, { "identity" : "swift-log", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-log.git", "state" : { "revision" : "5d66f7ba25daf4f94100e7022febf3c75e37a6c7", "version" : "1.4.2" } }, { "identity" : "swift-metrics", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-metrics.git", "state" : { "revision" : "1c1408bf8fc21be93713e897d2badf500ea38419", "version" : "2.3.1" } }, { "identity" : "swift-nio", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-nio.git", "state" : { "revision" : "124119f0bb12384cef35aa041d7c3a686108722d", "version" : "2.40.0" } }, { "identity" : "swift-nio-extras", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-nio-extras.git", "state" : { "revision" : "a75e92bde3683241c15df3dd905b7a6dcac4d551", "version" : "1.12.1" } }, { "identity" : "swift-nio-http2", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-nio-http2.git", "state" : { "revision" : "108ac15087ea9b79abb6f6742699cf31de0e8772", "version" : "1.22.0" } }, { "identity" : "swift-nio-ssl", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-nio-ssl.git", "state" : { "revision" : "c30c680c78c99afdabf84805a83c8745387c4ac7", "version" : "2.20.2" } }, { "identity" : "swift-nio-transport-services", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-nio-transport-services.git", "state" : { "revision" : "2cb54f91ddafc90832c5fa247faf5798d0a7c204", "version" : "1.13.0" } }, { "identity" : "swift-protobuf", "kind" : "remoteSourceControl", "location" : "https://github.com/apple/swift-protobuf.git", "state" : { "revision" : "e1499bc69b9040b29184f7f2996f7bab467c1639", "version" : "1.19.0" } } ], "version" : 2 } ```

If using CocoaPods, the project's Podfile.lock

No response

google-oss-bot commented 2 years ago

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

rizafran commented 2 years ago

Thanks for reporting, @vadimbelyaev. Upon checking the release notes, there were no changes related to uploading dSYMs that time. We would like to know if the action has enough memory? Note that the upload-symbols can use a good amount of memory on large dSYMs.

google-oss-bot commented 2 years ago

Hey @vadimbelyaev. We need more information to resolve this issue but there hasn't been an update in 5 weekdays. I'm marking the issue as stale and if there are no new updates in the next 5 days I will close it automatically.

If you have more information that will help us get to the bottom of this, just add a comment!

vadimbelyaev commented 2 years ago

@rizafran, thanks for looking into it and sorry for my delayed reply. Unfortunately I was not able to find any info about memory limits for GitHub actions. However, I've just tried to reproduce the issue with Xcode 14 and the latest Firebase iOS SDK 9.6.0 while having Crashlytics/run --debug in my build step and it all went successfully.

Since I'm no longer able to reproduce the issue, I'm closing it.

If it ever repeats, I'll try adding the verbose output of the run script to this issue.

Cheers! -Vadim

firebase / firebase-ios-sdk