smallint
commented
10 years ago Thanks a lot Stephan. Since associating pts with an output buffer as in #28 should not make a difference as long as the VPU reorders frames correctly than using a priority queue I would go for the current approach as in #28. The main reasons for that are:
- no additional dynamic data structure with is loosely associated with the output buffers
- no synchronization of that queue with output buffers
- no potential overrun of the queue in case of logic errors
koying
wolfgar
emveepee
Intro
I open this new thread following the discussion in #8 and the recent PR #28. The point is to share about the best solution to implement properly pts (presentation timestamp) association with output frames.
What's the question about ?
Encoded frames are pushed out of the demuxer in ::Decode function. The encoded unit is associated with a pts, at least for decent (ie no AVI) containers. We will have to keep proper association between this encoded data and the resulting output picture as the pts is useful to know when the final output pictures (retrieved in ::Getpicture) have to be displayed. Conceptually it is pretty simple but the VPU library does not provide any help to track these pts and we have to implement this logic by ourselves. During the initial implementation we relied upon a freescale library mfw_gst_ts which is provided with their gstreamer plugins. Even if this library has some assets (especially, when MODE_AI is selected and when functions with a '2' suffix are used) it also has some subtle issues that result in inappropriate time corrections (even when we really strive to push only pts that have to be queued) and it is considered overcomplicated for integration in this xbmc implementation.
One could think a simple FIFO would just be fine but it is not as simple :
Before explaining further the current implementation let's remind how VPU processes data : A set of buffers (contiguous in memory so that they can used for DMA) are allocated for VPU use. These buffers are used to store decoded pictures. When encoded data is submitted to the VPU to be processed we retrieve 2 indexes : an index for consumed frame and an index for display frame. If these indexes are positive value they point to a VPU buffer. This way of working is really the one which occurs at low level (at the interface with the DSP) If a consumed index is returned, the flag VPU_DEC_ONE_FRM_CONSUMED is set and if a display index is available the flag VPU_DEC_OUTPUT_DIS is set out of VPU_DecDecodeBuf function (this function is provided by libfslvpuwrap library) Note that, we operate the VPU so that it ensures display pictures are issued in order.
Back to pts handling, it whole idea of the current implementation is to associate the pts to the VPU buffer when the frame is consumed. That way we are sure that we will retrieve the correct pts associated to this buffer when the picture will be out for display. Also note that doing so will "naturally" reorder the pts at the same time as the display buffers are reordered. If is not the case it would be the proof that something goes wrong. And in fact this anomaly was witnessed with some livetv stream at some stage : This is exactly what PR #28 addresses.
As an alternative, an implementation based on an sorted queue was proposed in PR #8. The approach is of course perfectly valid : It addresses the 1st drawback of FIFO approach. Yet the 2 others items have still to be addressed to get a robust implementation based on sorted queue.
Conclusion
I hope that this little post can be helpful to clarify the whole pts subject and will help to decide what we want to do for the definitive implementation...