shogo4405 / HaishinKit.swift

Camera and Microphone streaming library via RTMP and SRT for iOS, macOS, tvOS and visionOS.
https://docs.haishinkit.com/swift/latest
BSD 3-Clause "New" or "Revised" License
2.79k stars 619 forks source link

60fps for IOStreamRecorder not working in version 1.9.6? #1635

Closed kenle closed 3 hours ago

kenle commented 1 week ago

Describe the bug

Hi @shogo4405, I can't get 60fps working in file record with version 1.9.6 for some reason.

In my old code using version 1.4.6 I was able to get 60fps.

To Reproduce

            var recorder = IOStreamRecorder()
            stream.addObserver(recorder!)
            stream.frameRate = 60;
            recorder?.delegate = self;

            recorder?.settings = [
                AVMediaType.audio: [
                    AVFormatIDKey: Int(kAudioFormatMPEG4AAC),
                    AVSampleRateKey: 0,
                    AVNumberOfChannelsKey: 0,
                    AVEncoderBitRateKey: 128000,
                ],
                AVMediaType.video: [
                    AVVideoCodecKey: AVVideoCodecType.hevc,
                    AVVideoHeightKey: 1920,
                    AVVideoWidthKey: 1080,
                    AVVideoCompressionPropertiesKey: [
                        AVVideoMaxKeyFrameIntervalDurationKey: 2,
                        AVVideoExpectedSourceFrameRateKey: 60,
                        AVVideoProfileLevelKey: kVTProfileLevel_HEVC_Main_AutoLevel,
                        AVVideoAverageBitRateKey: 10000 * 1000,
                    ] as [String : Any]
                ]
            ];

            recorder?.startRunning();

Expected behavior

Should get 60fps, however only getting 30fps.

Version

1.9.6

Smartphone info.

iPhone 12 Pro Max, iOS version 17.6.1

Additional context

Tracing around the code I think when I did stream.frameRate = 60 is where it is suppose to set it to 60 fps?

IOStream.swift
    public var frameRate: Float64 {
        get {
            return lockQueue.sync { self.mixer.videoIO.frameRate }
        }
        set {
            lockQueue.async {
                self.mixer.videoIO.frameRate = newValue
            }
        }
    }

IOMixer.swift
    static let defaultFrameRate: Float64 = 30

IOVideoUnit
    var frameRate = IOMixer.defaultFrameRate {
        didSet {
            guard #available(tvOS 17.0, *) else {
                return
            }
            for capture in captures.values {
                capture.setFrameRate(frameRate)
            }
        }
    }

IOVideoCaptureUnit
    func setFrameRate(_ frameRate: Float64) {
        guard let device else {
            return
        }
        do {
            try device.lockForConfiguration()
            if device.activeFormat.isFrameRateSupported(frameRate) {
                device.activeVideoMinFrameDuration = CMTime(value: 100, timescale: CMTimeScale(100 * frameRate))
                device.activeVideoMaxFrameDuration = CMTime(value: 100, timescale: CMTimeScale(100 * frameRate))
            } else {
                if let format = device.videoFormat(
                    width: device.activeFormat.formatDescription.dimensions.width,
                    height: device.activeFormat.formatDescription.dimensions.height,
                    frameRate: frameRate,
                    isMultiCamSupported: device.activeFormat.isMultiCamSupported
                ) {
                    device.activeFormat = format
                    device.activeVideoMinFrameDuration = CMTime(value: 100, timescale: CMTimeScale(100 * frameRate))
                    device.activeVideoMaxFrameDuration = CMTime(value: 100, timescale: CMTimeScale(100 * frameRate))
                }
            }
            device.unlockForConfiguration()
        } catch {
            logger.error("while locking device for fps:", error)
        }
    }

Screenshots

No response

Relevant log output

No response

shogo4405 commented 6 days ago

Yes, if the camera supports that value, the FPS setting will be applied to the camera. If you are using the offscreen rendering feature, additional settings like the ones below are also required.

if stream.videoMixerSettings.mode == .passthrough

stream.frameRate = 60

if stream.videoMixerSettings.mode == .offscreen

stream.frameRate = 60
stream.screen.frameRate = 60