multiple upload-symbols with 100% cpu usage

kenji21 commented 3 years ago

Step 0: Are you in the right place?

For issues or feature requests related to the code in this repository file a Github issue.
- If this is a feature request please use the Feature Request template.
For general technical questions, post a question on StackOverflow with the firebase tag.
For general (non-iOS) Firebase discussion, use the firebase-talk google group.
For backend issues, console issues, and other non-SDK help that does not fall under one of the above categories, reach out to Firebase Support.
Once you've read this section and determined that your issue is appropriate for this repository, please delete this section.

[REQUIRED] Step 1: Describe your environment

Xcode version: 12.2
Firebase SDK version: 7.1.0
Installation method: Swift Package Manager
Firebase Component: upload-symbols/run (Auth, Core, Database, Firestore, Messaging, Storage, etc)

[REQUIRED] Step 2: Describe the problem

Steps to reproduce:

What happened? How can we make the problem occur? Just built & run project test target ten times in a row (or maybe having designable in the project could do this)

If you have a downloadable sample project that reproduces the bug you're reporting, you will likely receive a faster response on your issue.

Relevant Code:

Screenshot 1

Ask people to tick the install only checkbox in the build phase, or better: determine if a build is an Archive one (uploading to the store... so we really need to upload symbols...) maybe like this (early exit for our linter script here):

if [[ "$BUILD_DIR" == *"IBDesignables"* ]] || [[ "$BUILD_DIR" == *"Previews"* ]] ; then
    echo "not linting for IBDesignables/SwiftUI Previews builds";
    exit 0
fi

As I was building for testing, App under unit test is correctly build, so the script phase was called

Screenshot 2

google-oss-bot commented 3 years ago

I couldn't figure out how to label this issue, so I've labeled it for a human to triage. Hang tight.

samedson commented 3 years ago

Hey @kenji21, are you trying this out on a really large app? upload-symbols should run for a fairly short time on small apps, but can take longer on larger apps.

The reason we don't tell devs to check the "For install builds only" option is because many devs want crashes to show up in development, and first time onboarding requires upload-symbols to succeed so developers can make sure everything is set up correctly.

Using the run script makes upload-symbols run in the background. If you'd like it to run synchronously you can use:

${PROJECT_DIR}/vendor/FirebaseCrashlytics/upload-symbols --build-phase

kenji21 commented 3 years ago

Sure the project is 15k lines of swift. I don't want my unit tests to be runned slower (neither IBInspectable/SwiftUI previews builds)... I think it is wasteful to upload symbols on each debug build... maybe you should just mention the checkbox in your README

kjyv commented 2 years ago

I apparently reported the same issue over here: https://github.com/firebase/firebase-unity-sdk/issues/81

However, with Unity, there is always a huge amount of il2cpp generated code and the issue is not so much the time it takes at 100% cpu but the amount of memory it fills. After a few seconds it fills all remaning memory available on my machine. It seems to not be limited to a resonable amount regarding how much memory the system has. It quickly makes the system unusable and even stalls the mouse cursor meaning upload-symbols can't do it's job because we have to kill it manually every time.

kenji21 commented 2 years ago

The reason we don't tell devs to check the "For install builds only" option is because many devs want crashes to show up in development, and first time onboarding requires upload-symbols to succeed so developers can make sure everything is set up correctly.

OK, then for "Debug builds", maybe you should check if the scheme has the debugger unticked (which is enabled by default), and Xcode will show the stack trace while debugging, so useless to upload symbols for such a build :

kjyv commented 2 years ago

The Xcode project is generated, so anything that I change will be overridden the next time. The issue is not that upload-symbols is run, the issue is that it takes too much memory and cpu for big projects. It also doesn't do that every time, sometimes it is done without me noticing.

kenji21 commented 2 years ago

It also doesn't do that every time, sometimes it is done without me noticing.

random issue, hardest one to fix, 100% cpu means an infinite loop, but why such a loop happens ? can it be caused by a clean of the build folder (e.g. dSYM disappearing from disk during upload) ?

kenji21 commented 2 years ago

maybe gathering a sample (from Activity Monitor) could give some clue

samedson commented 1 year ago

For folks running into this, is it the case that the tool is completing the upload eventually, or is it really stuck in an infinite loop?

kjyv commented 1 year ago

I'm no longer working on the project but what I've encountered was that it was staying at 100% for many minutes - waiting for it to finish was not realistically possible though as everything froze when the memory was used up. In order to reproduce, I recommend trying a default Unity sample project with Firebase added and just build it a few times.

samedson commented 1 year ago

It seems it may still be an issue in https://github.com/firebase/firebase-ios-sdk/issues/11159. I'm going to close this and continue the investigation in the new issue

firebase / firebase-ios-sdk