No camera preview on iOS 13

[SergejLogis]

Description

We are using Twilio Video in SwiftUI app. On iOS 14 everything works fine, but on iOS 13 local camera preview stays empty no matter what. There are no errors or warnings in console, both Camera Source and Camera Device are obtained successfully but VideoView stays empty.

Code

class TwilioRoomInteractor {

    private let cameraSourceSubject: CurrentValueSubject<CameraSource?, Never> = .init(nil)
    private let cameraTrackSubject: CurrentValueSubject<LocalVideoTrack?, Never> = .init(nil)

    func cameraTrack() -> AnyPublisher<(track: LocalVideoTrack, position: AVCaptureDevice.Position), Never> {
        return cameraTrackSubject
            .compactMap { $0 }
            .zip(
                cameraSourceSubject
                    .compactMap { $0?.device?.position }
            )
            .map { (track: $0, position: $1) }
            .eraseToAnyPublisher()
    }

    private func createLocalVideoTrack(cameraPosition: AVCaptureDevice.Position = .front) -> Bool {
        releaseCameraTrack()

        let options = CameraSourceOptions { builder in
            if let scene = SceneDelegate.scene {
                builder.orientationTracker = UserInterfaceTracker(scene: scene)
            }
        }

        guard
            let cameraSource = CameraSource(options: options, delegate: self),
            let cameraDevice = CameraSource.captureDevice(position: cameraPosition)
        else {
            log.error(in: .webRTC, "Failed to initialize capture device at %@!", "\(cameraPosition)")
            return false
        }

        let cameraTrack = LocalVideoTrack(
            source: cameraSource,
            enabled: true,
            name: "\(cameraPosition == .front ? "Front" : "Back") camera"
        )

        // Start camera capture.
        cameraSource.startCapture(device: cameraDevice) { [weak self] _, _, error in
            if let error = error {
                log.error(in: .webRTC, "Camera capture failed with error: %@", error.localizedDescription)

            } else {
                self?.cameraSourceSubject.value = cameraSource
                self?.cameraTrackSubject.value = cameraTrack
            }
        }

        return true
    }

    /// Stops camera capture and releases camera video track.
    private func releaseCameraTrack() {
        if let cameraTrack = cameraTrackSubject.value {
            cameraTrack.isEnabled = false
            for renderer in cameraTrack.renderers {
                cameraTrack.removeRenderer(renderer)
            }
        }
        cameraTrackSubject.value = nil

        cameraSourceSubject.value?.stopCapture()
        cameraSourceSubject.value = nil
    }

    ...
}

class UITwilioCameraPreviewView: UIView {

    private weak var videoView: VideoView?

    private let interactor: TwilioRoomInteractorProtocol
    private var cancelBag = CancelBag()

    private func initialize() {
        let videoView = setupVideoView()

        interactor.cameraTrack().sink { cameraTrack, cameraPosition in
            let command = {
                cameraTrack.removeRenderer(videoView)
                cameraTrack.addRenderer(videoView)
                videoView.shouldMirror = (cameraPosition == .front)
            }

            if videoView.hasVideoData {
                UIView.transition(
                    with: videoView,
                    duration: Animation.Duration.default,
                    options: [
                        .curveLinear,
                        (cameraPosition == .front ? .transitionFlipFromRight : .transitionFlipFromLeft)
                    ],
                    animations: command,
                    completion: nil
                )
            } else {
                command()
            }
        }.store(in: cancelBag)

        interactor.onDisconnect().sink { [weak self] in
            self?.dismantle()
        }.store(in: cancelBag)
    }

    private func setupVideoView() -> VideoView {
        let videoView = VideoView(frame: frame)
        videoView.contentMode = .scaleAspectFill
        addSubview(videoView)
        self.videoView = videoView

        NSLayoutConstraint.activate([
            videoView.widthAnchor.constraint(equalTo: widthAnchor),
            videoView.heightAnchor.constraint(equalTo: heightAnchor),
            videoView.centerXAnchor.constraint(equalTo: centerXAnchor),
            videoView.centerYAnchor.constraint(equalTo: centerYAnchor)
        ])

        return videoView
    }

    ...

Expected Behavior

Camera preview outputs video.

Actual Behavior

Camera preview stays empty.

Reproduces How Often

100% of times, but only on iOS 13

Versions

Video iOS SDK

TwilioVideo 3.7.1

Xcode

Xcode 12.2

iOS Version

iOS 13.7

iOS Device

iPhone 8, iPhone X

Any help would be greatly appreciated 🙏

[Hamoonist]

With iOS 14 one of my devices are like this. So far in my case I have pinpointed the problem to cameraSource.startCapture(device: cameraDevice, completion:) The method is called, but the completion never. Can you try and put an Xcode break or a print statement inside the completion to see if your problem is the same?

[piyushtank]

The VideoView renderer is a UIView but we haven't tried its integration with SwiftUI yet. We have a task in our backlog to write a sample app with SwiftUI which we plan to address soon.

We will update this ticket when we have more information.

[SergejLogis]

@Hamoonist unfortunately, I don't have iOS 13 device on hands, and it's impossible to test camera on simulator...

@piyushtank Twilio works like charm with SwiftUI. We have zero issues with iOS 14. However, on iOS 13 the camera preview doesn't work so it's pretty much a deal breaker if you must support iOS 13.

[Hamoonist]

So the problem we had with cameraSource.startCapture(device: cameraDevice, completion:) appeared out of nowhere, and got resolved by itself! I wanted to share the good news 😅