FlipCamera Lag - Githubissues

ksuhr1 commented 3 years ago

Description

Hi there! I have one issue: cameraLag with calling flipCamera. I have two view controllers for two different users: Designer and Client. On Designer side I just want to flip the camera like normally. I have a button on my UI where I call flipCamera. I notice that when I switch the camera for both view controllers, the frame flips right before the camera capture actually switches so it looks buggy.

On the Client side I am rendering frames using the ExampleSampleBuffer view example project and flip the frame using OpenCv flip function when the camera position is in the front. On the Designer side I render the frames the same as video-quickstart example project.

It has the problem of the frame flipping before the camera capture actually switches. If you have any advice on how to prevent the frame from flipping before the camera actually switches I'd really appreciate it!

Steps to Reproduce

Press Switch Camera multiple times
Should see frame flipping quickly before camera capture actually flips

Code

Designer View Controller

 @IBAction func switchCamera(_ sender: Any) {
         self.flipCamera()

 }

 @objc func flipCamera() {
    var newDevice: AVCaptureDevice?

    if let camera = self.camera, let captureDevice = camera.device {
        if captureDevice.position == .front {
            newDevice = CameraSource.captureDevice(position: .back)
        } else {
            newDevice = CameraSource.captureDevice(position: .front)
        }

        if let newDevice = newDevice {
            camera.selectCaptureDevice(newDevice) { (captureDevice, videoFormat, error) in
                if let error = error {
                    self.logMessage(messageText: "Error selecting capture device.\ncode = \((error as NSError).code) error = \(error.localizedDescription)")
                } else {
                    self.editPreviewView.shouldMirror = (captureDevice.position == .front)
                }
            }
        }
    }
}

Client View Controller

 @objc func flipCamera() {
    print("flipCamera")
    TwilioVideoSDK.setLogLevel(.debug)
    var newDevice: AVCaptureDevice?
        if let camera = self.camera, let captureDevice = camera.device {

            if captureDevice.position == .front {
                newDevice = CameraSource.captureDevice(position: .back)
                self.isFrontOn = false
                self.flashOn = true

            } else {
                newDevice = CameraSource.captureDevice(position: .front)
                self.isFrontOn = true
                self.flashOn = false
            }
            if let newDevice = newDevice {
                camera.selectCaptureDevice(newDevice) { (captureDevice, videoFormat, error) in
                    if let error = error {
                        self.logMessage(messageText: "Error selecting capture device.\ncode = \((error as NSError).code) error = \(error.localizedDescription)")
                    }
            }
        }
    }
}

Expected Behavior

When I click the switch Camera button, I want the camera to flip without inverting the frame right before it does that.

Actual Behavior

When I switch the camera for both view controllers, the frame flips right before the camera capture actually switches so it looks buggy.

Reproduces How Often

Happens every time I press the switch button after a few button clicks, not set amount, kind of random.

Xcode

[11.7]

iOS Version

[13.5.1]

iOS Device

[11 Pro Max]

piyushtank commented 3 years ago

@ksuhr1 Thanks for reaching out - what you are observing might be an underlying synchronization bug between renderer and capturer. @ceaglest is out-of-office right now, once he is back, we will get back to you on this.