Where to attach a live buffer

Thanks for this great library, it enables me to do totally new stuff in browser applications. One thing i didnt get is how i could implement a little buffer that would take care about synchronizing a/v by looking at pts and also solve stuttering issues when the source encoder does not deliver the packets in the expected frequency but instead in bursts (like vlc does for some source formats).

The base idea is to save either packets or decoded A/V buffers and serve them in a loop that runs in the analyzed framerate frequency and outputs only the packets for the next 1-frame period (maybe a little more audio, depends on audio frame size?)

What i tried is to alter ts demuxer source code but i was under the impression that this is the wrong place. E.g. i saved the first packet and altered packetComplete to output only this packet but the result was totally unexpected: 1 second repeated (instead of 1 frame), sometimes randomly playing even 2 seconds loop.

As i was working with i-frame only, i am not sure what exactly was happening, i mean i saved the first packet, did it contain more than one frame? (from reading the source code, i don't think that the mpeg decoder expects more than 1 video frame in a packet). Also, of course i tried to play with different ts encoding options like placing PAT/PMT after every frame but it didnt change.

So the question is, is it possible and where would be the correct place to hook in?

Seems like the tsdemuxer is the right place to attach a buffer. What i needed to do in order to make this work is to make a copy of the "pi" in packetComplete instead of buffering the demuxed pi directly. I have the feeling that the buffers of pi's are being reused and overwritten internally in jsmpeg in some way.

It works as a proof of concept but i am not yet sure if i prefer this way or let ffmpeg do the buffering by piping vlc output through ffmpeg using -re -i - option. I guess i will go for the second option because i am not yet convinced that my interval based rendering method is easy to maintain and easy to develop in a way that it's compatible to all source formats (audio only etc..)

TS.prototype.copyPi = function(pi){
    var newPi = {};
    newPi.destination = pi.destination;
    newPi.pts = pi.pts;
    newPi.totalLength = pi.totalLength;
    newPi.currentLength = pi.currentLength;
    newPi.buffers = pi.buffers.slice();
    return newPi;
}

phoboslab / jsmpeg

Where to attach a live buffer #405