Xcode15.4+, High CPU Usage and Increse Memory with SPM

[RainyTunes]

Describe the bug

I found that RAM constantly increases when running on 1.9.1 version However, it doesn't happen on 1.8.x version.

To Reproduce

In my project, uncomment the line, build the project and do nothing after launching the app, then, Memory will constantly increases in the built project

If it remains uncommented, the issue never happens.

It seems that something related to Memory Leak exists in the new version.

 lazy var stream: RTMPStream = {
        HUBLog("stream init")
        let stream = RTMPStream(connection: connection)
        stream.frameRate = 30.0
        stream.sessionPreset = .hd1280x720
        stream.videoMixerSettings.mode = .offscreen
        stream.screen.size = .init(width: 1280, height: 720)
        stream.screen.backgroundColor = UIColor.white.cgColor
//        stream.screen.startRunning()
        stream.isMultiTrackAudioMixingEnabled = true
        _ = stream.registerVideoEffect(backgroundVideoEffect)
        return stream
    }()

Expected behavior

No Memory Leak

Version

1.9.1

Smartphone info.

No response

Additional context

No response

Screenshots

No response

Relevant log output

No response

[RainyTunes]

By the way, I get severe hangs when my Xcode-debugging app cold starts. For each App launch, they last for half a minute and UI freezes, then suddenly dismiss like they never exist. The release version is okay.

It is not a HaishinKit issue, but it appears recently, about after I take your suggestion to update my Xcode to 15.4. So I would like to ask you do you have any ideas about this? Is this an Xcode issue?

[floriangbh]

Same issue here with v1.9.1and XCode 15.4 :

[shogo4405]

Unfortunately, I could not reproduce the issue on my end. I am testing with iOS 17.5.1 and Xcode 15.4.

1. It seems there is no memory leak with the Example app.
2. I tried initializing RTMPStream as instructed, but the issue persists.
   diff

diff --git a/Examples/iOS/IngestViewController.swift b/Examples/iOS/IngestViewController.swift
index de8f7770..561b101d 100644
--- a/Examples/iOS/IngestViewController.swift
+++ b/Examples/iOS/IngestViewController.swift
@@ -23,24 +23,37 @@ final class IngestViewController: UIViewController {
     private var retryCount: Int = 0
     private var preferedStereo = false
     private let netStreamSwitcher: NetStreamSwitcher = .init()
-    private var stream: IOStream {
-        return netStreamSwitcher.stream
-    }
+
+    private lazy var connection: RTMPConnection = {
+        return RTMPConnection()
+    }()
+
+    private lazy var stream: RTMPStream = {
+        let stream = RTMPStream(connection: connection)
+        stream.frameRate = 30.0
+        stream.sessionPreset = .hd1280x720
+        stream.videoMixerSettings.mode = .offscreen
+        stream.screen.size = .init(width: 1280, height: 720)
+        stream.screen.backgroundColor = UIColor.white.cgColor
+        stream.screen.startRunning()
+        stream.isMultiTrackAudioMixingEnabled = true
+        // _ = stream.registerVideoEffect(backgroundVideoEffect)
+        return stream
+    }()
+
     private lazy var audioCapture: AudioCapture = {
         let audioCapture = AudioCapture()
         audioCapture.delegate = self
         return audioCapture
     }()
     private var videoScreenObject = VideoTrackScreenObject()
+    private var recorder = IOStreamRecorder()

     override func viewDidLoad() {
         super.viewDidLoad()

         netStreamSwitcher.uri = Preference.default.uri ?? ""
-
-        stream.videoMixerSettings.mode = .offscreen
-        stream.screen.size = .init(width: 720, height: 1280)
-        stream.screen.backgroundColor = UIColor.white.cgColor
+        stream.addObserver(recorder)

         videoScreenObject.cornerRadius = 16.0
         videoScreenObject.track = 1
@@ -136,7 +149,11 @@ final class IngestViewController: UIViewController {
     }

     @IBAction func toggleTorch(_ sender: UIButton) {
-        stream.torch.toggle()
+        if recorder.isRunning.value {
+            recorder.stopRunning()
+        } else {
+            recorder.startRunning()
+        }
     }

     @IBAction func on(slider: UISlider) {
@@ -174,11 +191,12 @@ final class IngestViewController: UIViewController {
     @IBAction func on(publish: UIButton) {
         if publish.isSelected {
             UIApplication.shared.isIdleTimerDisabled = false
-            netStreamSwitcher.close()
+            connection.close()
             publish.setTitle("●", for: [])
         } else {
             UIApplication.shared.isIdleTimerDisabled = true
-            netStreamSwitcher.open(.ingest)
+            connection.addEventListener(.rtmpStatus, selector: #selector(rtmpStatusHandler), observer: self)
+            connection.connect(Preference.default.uri!)
             publish.setTitle("■", for: [])
         }
         publish.isSelected.toggle()
@@ -261,6 +279,22 @@ final class IngestViewController: UIViewController {
         }
     }

+    @objc
+    private func rtmpStatusHandler(_ notification: Notification) {
+        let e = Event.from(notification)
+        guard let data: ASObject = e.data as? ASObject, let code: String = data["code"] as? String else {
+            return
+        }
+        logger.info(code)
+        switch code {
+        case RTMPConnection.Code.connectSuccess.rawValue:
+            (stream as? RTMPStream)?.publish(Preference.default.streamName!)
+            retryCount += 1
+        default:
+            break
+        }
+    }
+
     @objc
     private func didInterruptionNotification(_ notification: Notification) {
         logger.info(notification)

It is highly likely that the issue occurs with the combination of the code written on the your side and version 1.9.1. Could you please provide the minimal code that can reproduce the issue?

[RainyTunes]

After spending several days investigating a persistent memory leak, I finally traced the issue back to my own code. Interestingly, this leak only became apparent after updating to HaishinKit 1.9.1. So I still believe this version prone to cause memory leaks, for example, when stream.videoSettings.mode = .offscreen.

To summarize, I wanted to share my findings with the community as it might help others facing similar issues. I will be closing this issue since the root cause has been identified within my own code.

@floriangbh @shogo4405

[RainyTunes]

I am sorry to reopen this issue because the memory leak happens again, without expectation. I spent one more day resolving this problem, but I cannot locate where the true error is.

What I have found up to now:

1. I cannot reproduce in the Demo either.

2. The problem doesn't happen in my project with 1.8.3, but it happen in my project with 1.9.1, with the same integration code. The RAM starts to increase once I call attachStream().

3. I tried to delete all other unnecessary code and kept only HaishinKit integration code, and even I didn't start to public stream, the problem still existed

4. For every try, the RAM increases regularly, about 0.3-0.6MB per second. This is very big increase and I guess it indicates the problem is related to video frame data within HaishinKit. WIth Xcode Memory Graph, it seems that numerous NSMutableArray are constantly being created. I believe it cannot be casued by my code side because I deleted nearly all other business code during debugging.

5. When I mannually set the RTMPStream to nil, RAM starts to constantly decrease, but with a very slow pace.

(But when I tap the leaks filter button, Memory Graph doesn't give much information)

Although I cannot 100 percent judge this is a HaishinKit issue, but I hope my information can bring some clues. I will continue to investigate on this problem later.

[floriangbh]

@RainyTunes same case for me, view.attachStream(self.rtmpStream) start the leak.

Leak information :

This is not present in the demo project, I will continue to investigate.

[RainyTunes]

@RainyTunes same case for me, view.attachStream(self.rtmpStream) start the leak.

Leak information :

This is not present in the demo project, I will continue to investigate.

I have similar Memory Graph diagram with yours, but I have no idea what these objects are.

It was really strange, because when I made a few modifications several days ago , and then the memory leak disappeared. That caused me to make an incorrect temporary conclusion at that time. But now, the memory leak came again and I cannot solve it this time.

[floriangbh]

After checking I also have a small leak with 1.7.1 ! It seems more related to my code + Xcode/Swift version which is now more sensible to leak.

[RainyTunes]

After checking I also have a small leak with 1.7.1 ! It seems more related to my code + Xcode/Swift version which is now more sensible to leak.

I agree with what you said "more sensible to leak."

However I have spend days optimizing incorrect retain circle and testing with deletion with unnecessary code, it seems very hard for me to find the source reason causing the leak.

[floriangbh]

v1.7.1 with Xcode 15.2 : OK v1.7.1 with Xcode 15.4: Leak

v1.9.2 with Xcode 15.4: Leak (cannot test with 15.2, v1.9.2 can't compile with Swift version)

[shogo4405]

I believe reproducing it on my end is the fundamental solution. To confirm the reproduction method, please let me know the following information: It would be helpful to create a project that is close to the one where the issue is occurring. Please provide as much information as possible.

1. iOS version you are using
2. Device model
3. Xcode version
4. Library integration method
   • [ ] CocoaPods?
   • [ ] SPM?
   • [ ] Carthage?
5. Project base
   • [ ] SwiftUI
   • [ ] UIKit
6. Type of view you are using
   • [ ] MTHKView
   • [ ] PiPHKView
7. Debug type
   • [ ] WiFi
   • [ ] mac connect

Additionally, it seems you are using effects, but does the issue persist if you turn them off?

_ = stream.registerVideoEffect(backgroundVideoEffect)

[shogo4405]

Does the same kind of leak occur in the Xcode 16 beta?

Xcode 16 is coming soon, and everyone will be busy with iOS 18 support. Please make it possible to build with versions 15.3 and 15.2. I was also considering the option of giving up on 15.4

[RainyTunes]

1. iOS version you are using iOS 18.0 and iPad 18.0

2. Device model iPhone 14 pro and iPad Air3

3. Xcode version Xcode 15.4

4. Library integration method SPM

5. Project base SwiftUI

6. Type of view you are using MTHKView

7. Debug type can reproduce on both WiFi and mac connect

8. Additionally, it seems you are using effects, but does the issue persist if you turn them off? YES

[RainyTunes]

Does the same kind of leak occur in the Xcode 16 beta?

Xcode 16 is coming soon, and everyone will be busy with iOS 18 support. Please make it possible to build with versions 15.3 and 15.2. I was also considering the option of giving up on 15.4

I have Xcode 16 Beta on my another mac, and the memory leak can reproduce too on Xcode 16 Beta. Thank you very much for your support.

[shogo4405]

Hello. Does the same issue occur with this repository as well? https://github.com/shogo4405/HaishinKit-MemoryLeak-Investigation HaishinKit-MemoryLeak-Investigation-main.zip

[floriangbh]

Hello @shogo4405,

When I run your project with Xcode 15.4 I can see a memory leak :

Another point If it can help with my project : v1.7.1 with Xcode 15.2 : CPU usage is stable at 43% v1.7.1 with Xcode 15.4: CPU usage jump to 115% and the phone overheat

I see the same CPU usage with your leak investigation project :)

[shogo4405]

Ah, thank you for the report.

I was also concerned about the CPU usage. Additionally, The situation is different, but... I observed a significant increase in memory with iPadOS 18 + Xcode 16.0.

On the other hand, with iPadOS 18 + Xcode 16.0, the demo version did not show any memory leaks at around 20% CPU usage. I think there must be some differences, so I would like to continue investigating.

https://github.com/shogo4405/HaishinKit.swift/tree/main/Examples/iOSSwiftUI

[shogo4405]

Hi. Can I see your Xcode's Manage scheme options?

Please turn off the Thread Performance Checker if it is currently on and check if it improves the situation.

Scheme	Memory

[floriangbh]

Hello,

After some tests on my project, I observed the same result as you, turning off the Thread Performance Checker improves the situation (no more memory leak, and CPU usage is OK!

[RainyTunes]

Hi. Can I see your Xcode's Manage scheme options?

Please turn off the Thread Performance Checker if it is currently on and check if it improves the situation.

Scheme Memory ![スクリーンショット 2024-07-28 21 35 38](https://github.com/user-attachments/assets/8fa8ba19-5f80-4d4a-b86c-4d287846e294ive

After following your guide, the issue has been resolved!

Thank you for being so support again. I think it would be great if the guide could be put in the README, as other developers may encounter the same issue

[RainyTunes]

By the way, could you please share how you identified that the Xcode Thread Performance Checker caused this issue? @shogo4405 @floriangbh

I would love to learn about your troubleshooting approach.