Agora v4, External Video Source, iOS

[mgrms]

Hello,

We are in the process of migrating from Agora v3 to v4. We used custom video source with MediaIO before, and now we need to migrate to Push method since MediaIO has been removed from SDK.

Since there is no complete example on how to use Push method in conjunction with custom AVFoundation camera, we have following questions:

1. As far as I understand, when using external video source, startPreview() functions will not have any effect for us, am I right? We need to manage (start/stop) preview manually using our custom video source?

2. After we setup our custom AVCaptureSession and start receiving CVPixelBuffers from AVCaptureSessionDelegate, can we start calling agora.pushExternalVideoFrame(videoFrame) right away? Or we must call this SDK function only when user is inside a channel?

To better demonstrate what I mean:

Variant 1:

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
        guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }

        let time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)

        let videoFrame = AgoraVideoFrame()
        videoFrame.format = 12
        videoFrame.textureBuf = pixelBuffer
        videoFrame.rotation = Int32(rotation)
        videoFrame.time = timeStamp
        agora.pushExternalVideoFrame(videoFrame)
    }

Variant B:

func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {
        guard let pixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }

        let time = CMSampleBufferGetPresentationTimeStamp(sampleBuffer)

        if isInAgoraVideoChannelRightNow { // HERE IS THE DIFFERENCE ====
            let videoFrame = AgoraVideoFrame()
            videoFrame.format = 12
            videoFrame.textureBuf = pixelBuffer
            videoFrame.rotation = Int32(rotation)
            videoFrame.time = timeStamp
            agora.pushExternalVideoFrame(videoFrame)
        }
    }

Or in other words, can we call/do we need to call agora.pushExternalVideoFrame(videoFrame) during local video preview, when user see the camera, but not in a channel yet? Which variant is correct? Does it make any difference in terms of performance, pricing, usage calculation?

Thanks.

[plutoless]

As far as I understand, when using external video source, startPreview() functions will not have any effect for us, am I right? We need to manage (start/stop) preview manually using our custom video source? Yes. Correct.

After we setup our custom AVCaptureSession and start receiving CVPixelBuffers from AVCaptureSessionDelegate, can we start calling agora.pushExternalVideoFrame(videoFrame) right away? Or we must call this SDK function only when user is inside a channel? You can start calling api right away. You don't have to wait until you join a channel.

Hope this helps.

[mgrms]

@plutoless

Thank you very much for your answer! I'll close this now.