tucan9389 / TFLiteSwift-Vision

Apache License 2.0
38 stars 1 forks source link

Assertion Error with assert(sourcePixelFormat == kCVPixelFormatType_32BGRA) #42

Open patrickunterwegs opened 2 years ago

patrickunterwegs commented 2 years ago

Hello, first of all thank you for the great work on this library!

I have an issue and I hope you can help me track down the actual problem and how to solve it. Basically I am not that experienced, but I managed to create a CVPixelBuffer and passed it on to the ModelDataHandler which is basically a copy of https://github.com/tucan9389/TFLiteSwift-Vision/blob/master/Examples/object_detection/ObjectDetection/ModelDataHandler/ModelDataHandler.swift

I am using the following code:

    // camera model
    class CameraModel: NSObject, ObservableObject, AVCaptureVideoDataOutputSampleBufferDelegate {

       ...

        func checkPermission() {
          ...
        }

        func setUp() {
           ...
        }

        func captureOutput(_ output: AVCaptureOutput, didOutput sampleBuffer: CMSampleBuffer, from connection: AVCaptureConnection) {

            guard let pixelBuffer: CVPixelBuffer = CMSampleBufferGetImageBuffer(sampleBuffer) else { return }

            let handler = ModelDataHandler(
                modelFileInfo: FileInfo(name: "object_labeler", extension: "tflite"),
                labelsFileInfo: FileInfo(name: "mobile_object_labeler_v1_labelmap", extension: "csv"))
...

This is actually pretty straightforward (once it's done at least).

But when I start the camera view, the app stops with an error in TFLiteVisionInput - the assertion fails.

    func croppedPixelBuffer(with inputModelSize: CGSize, and cropType: TFLiteVisionInterpreter.CropType, from targetSquare: CGRect? = nil) -> CVPixelBuffer? {
        guard let pixelBuffer = pixelBuffer else { return nil }
        let sourcePixelFormat = CVPixelBufferGetPixelFormatType(pixelBuffer)
        assert(sourcePixelFormat == kCVPixelFormatType_32BGRA)

Do you have any hint where this could be resolved?