search

shogo4405 / HaishinKit.swift

Camera and Microphone streaming library via RTMP and SRT for iOS, macOS, tvOS and visionOS.
BSD 3-Clause "New" or "Revised" License
2.76k stars 612 forks source link

VideoEffect not executing in HaishinKit 1.9.3 #1568

Closed jhays closed 3 weeks ago

jhays commented 3 weeks ago

Describe the bug

After updating to HaishinKit 1.9.3 in a project, I observed that a feature that uses VideoEffect was no longer working. The video frames in the camera preview remain unchanged even when the effect has been registered. While debugging, I added breakpoints within the VideoEffect subclass's execute func, and I found that this breakpoint is never hit.

I then tested the Example project within HaishinKit 1.9.3 release, and found the same result. It appears that VideoEffect is not being executed.

To Reproduce

  1. Run HaishinKit 1.9.3 Example (I tested the SwiftUI example)
  2. Place a breakpoint inside the execute functions inside bothPronamaEffect and MonochromeEffect
  3. Tap the on screen segmented control to select either of these video effects
  4. Observe that the breakpoints are never hit, and the camera preview does not show any filter or effect to the image.

Expected behavior

The VideoEffect filters, when enabled, should be running and applying filters to the frames of the camera preview.

Version

1.9.3

Smartphone info.

Device: iPhone 15 Pro Max OS: iOS 17.6.1

Additional context

No response

Screenshots

No response

Relevant log output

No response

shogo4405 commented 3 weeks ago

The issue with SwiftUI was my migration mistake. I have fixed it in this commit: 6282e82. To apply it from version 1.9.x, the following set is required. I have updated the migration guide. https://github.com/shogo4405/HaishinKit.swift/wiki/Moving-from-1.8.x-APIs-to-1.9.0#apply-videoeffect

スクリーンショット 2024-09-12 20 30 01

jhays commented 3 weeks ago

Great, thanks. I've tested this change and filters are working for me in the preview in SwiftUI.

Would the same change be needed if using SRT as well (on the SRTStream object)?

Thanks, this issue can be closed.

shogo4405 commented 3 weeks ago

Yes. A setup similar to SRTStream is required.

jhays commented 3 weeks ago

Shoot, unfortunately I've noticed a new problem. I found that when I set the AVCaptureDevice activeFormat to one that supports 60fps, if I include those two lines you suggested to fix the visual effect filters, my output is not going past 30 fps.

If I remove those two lines you suggested, the stream is able to reach 60fps.

This issue occurs even if I have video effects disabled.

Any idea why the videoMixerSettings.mode or screen.startRunning() could be capping my output fps to 30?