Quicktime crashes during recording

[PhilippMatthes]

TL;DR

The timestamp computation seems to be off, which leads to Quicktime not being able to record the virtual camera stream.

Long Version

I spun off a one-day project from yours to create a virtual webcam, which detects objects from the real webcam stream and shows them as boxes (with labels) embedded in the original webcam stream.

During development, I encountered a problem, which lead to the stream not being able to be recorded by the Quicktime App. You can reproduce this problem by opening the Quicktime app, selecting the virtual camera (SimpleDALPlugin) and clicking the record button. The expected behaviour was, that the stream is simply recorded and saved to a video. The actual behaviour, however, is that the recording immediately stops itself.

I was able to locate the core issue behind this problem, which lies behind this timestamp computation:

https://github.com/seanchas116/SimpleDALPlugin/blob/ddab598bac239b129d7de702346e7d5aab02663e/SimpleDALPlugin/Stream.swift#L135-L141

This timestamp is embedded into the posted stream. However, there seems to be a problem in this computation, which makes the timing in the resulting stream erroneous.

Because, in my fork, I use the real webcam with an AVCaptureOutput, I can receive this timing info like this:

extension VideoCapture: AVCaptureVideoDataOutputSampleBufferDelegate {
    public func captureOutput(
        _ output: AVCaptureOutput,
        didOutput sampleBuffer: CMSampleBuffer,
        from connection: AVCaptureConnection
    ) {
        var sampleTimingInfo = CMSampleTimingInfo()
        guard CMSampleBufferGetSampleTimingInfo(
            sampleBuffer,
            at: 0,
            timingInfoOut: &sampleTimingInfo
        ) == noErr else {return}
        // ...

(See the original code here)

... and pipe it to the stream, where it is simply reused and written into the stream, which leads to the original problem being mitigated, i.e. the virtual camera being able to be recorded by the Quicktime app.

Conclusion: the timestamp computation needs to be re-evaluated for correctness.

[iKenndac]

@PhilippMatthes Could you share the fields of a working CMSampleTimingInfo from the webcam (after you call CMSampleBufferGetSampleTimingInfo to fill it out)? I suspect that will help.

[PhilippMatthes]

@iKenndac I am currently busy, but you could also pull my project and build it yourself, then log the properties of the object and view the logs with the console app, to obtain the information behind the fields.

[seanchas116]

I tried it and got this:

CMSampleTimingInfo(
  duration: __C.CMTime(value: 1034, timescale: 30000, flags: __C.CMTimeFlags(rawValue: 1), epoch: 0),
  presentationTimeStamp: __C.CMTime(value: 962710847, timescale: 30000, flags: __C.CMTimeFlags(rawValue: 3), epoch: 0),
  decodeTimeStamp: __C.CMTime(value: 962710847, timescale: 30000, flags: __C.CMTimeFlags(rawValue: 3), epoch: 0))

duration is always same (1034) and decodeTimeStamp is same as presentationTimeStamp.

[seanchas116]

I managed to fix the issue in my environment: #3.

[PhilippMatthes]

@seanchas116 looks good, I can confirm that this fixes the issue.