Expose VideoFrame properties

[hiroshihorie]

Exposes buffer, pixelBuffer etc.

[sameers27]

Also can we make public toRTCType() on VideoFrame?

[hiroshihorie]

We don't expose WebRTC types directly, what is your use case ?

[sameers27]

We don't expose WebRTC types directly, what is your use case ?

We currently have our own RTCVideoFrame decoder. To continue using it we would need to convert the VideoFrame to RTCVideoFrame

[hiroshihorie]

Your RTCVideoFrame decoder is written for another WebRTC SDK ? Even I expose underlying WebRTC symbol directly it's a LKRTCVideoFrame object I don't think it's compatible to use directly compiler will complain.

What I think will work is to write a converter from VideoFrame to RTCVideoFrame. There shouldn't be much overhead since buffer memory isn't being moved around.

I can write an example snippet.

[sameers27]

Your RTCVideoFrame decoder is written for another WebRTC SDK ? Even I expose underlying WebRTC symbol directly it's a LKRTCVideoFrame object I don't think it's compatible to use directly compiler will complain.

What I think will work is to write a converter from VideoFrame to RTCVideoFrame. There shouldn't be much overhead since buffer memory isn't being moved around.

I can write an example snippet.

That would be great! Our decoder is written for any webrtc sdk. We also use it for livekit

[hiroshihorie]

Can you try something like :

extension LiveKit.VideoRotation {
    func toWebRTCRotation() -> WebRTC.RTCVideoRotation {
        switch self {
        case ._0: return ._0
        case ._90: return ._90
        case ._180: return ._180
        case ._270: return ._270
        }
    }
}

extension LiveKit.VideoFrame {
    // Example to convert frame
    func toWebRTCVideoFrame() -> WebRTC.RTCVideoFrame {
        let rtcBuffer: RTCVideoFrameBuffer
        if let buffer = buffer as? CVPixelVideoBuffer {
            rtcBuffer = WebRTC.RTCCVPixelBuffer(pixelBuffer: buffer.pixelBuffer)
        } else if let buffer = buffer as? I420VideoBuffer {
            rtcBuffer = WebRTC.RTCI420Buffer(width: buffer.chromaWidth,
                                             height: buffer.chromaHeight,
                                             dataY: buffer.dataY,
                                             dataU: buffer.dataU,
                                             dataV: buffer.dataV)
        } else {
            fatalError("Unsupported type")
        }

        return RTCVideoFrame(buffer: rtcBuffer,
                             rotation: rotation.toWebRTCRotation(),
                             timeStampNs: timeStampNs)
    }
}