Silence-GitHub / BBMetalImage

A high performance Swift library for GPU-accelerated image/video processing based on Metal.
MIT License
988 stars 126 forks source link

Video doesn't play in real time #28

Closed pennywise94 closed 4 years ago

pennywise94 commented 4 years ago

I use a metalView to show the camera, and after recording I use the same metalView to play back the video that was recorded to allow editing by the user.

videoWriter.finish {
    self.camera.isPaused = true
    self.videoSource = BBMetalVideoSource(url: self.outputURL)
    self.videoSource.playWithVideoRate = true
    self.videoSource.add(consumer: self.metalView!)
    self.videoSource.start { [weak self] (_) in
         guard let self = self else { return }
     }
}

I set playWIthVideoRate to true, tried before and after adding the metalView as consumer. So I don't know why this is happening. Any ideas?

Silence-GitHub commented 4 years ago

I copy the code and paste to the demo project. It works well. I suggest checking the recorded video file first. Play the video file with AVPlayer, or save to Photos and play (or share to Mac and play). If the video file is not good, we should check the code for recording.

pennywise94 commented 4 years ago

The video file is OK. I actually save the video to camera roll where it plays at normal, recorded speed.

Silence-GitHub commented 4 years ago

I guess the video plays faster. Check the frame time when playing. Make sure the frame time is consistent with the video file play rate (and the camera FPS when recording).

videoSource.start(progress: { time in
    print(time)
}, completion: nil)

If the frame time is OK. Check whether the code in BBMetalVideoSource.swift file at 235 line is executed. If this line is not executed, the video plays fast.

usleep(UInt32(1000000 * sleepTime))
pennywise94 commented 4 years ago

I actually found the solution just now. I use an audioDelegate to sync video and audio in the metalView, but because I hadn't initialized the audioPlayer yet, there was nothing to sync. So this is fixed!