Support for WebCodecs

[animeshk874]

Is there a way to make ts-demuxer work with new browser features like WebCodecs and Streams API? As mentioned in this article from web.dev, three parameters are needed to create an EncodedVideoChunk instance.

• A BufferSource of encoded video data
• the chunk's start timestamp in microseconds (media time of the first encoded frame in the chunk)
• the chunk's type (possible values - 'key' or 'delta')

As of now, ts-demuxer gives the BufferSource, but not the timestamp and the chunk's type. Is it possible to determine/calculate these two values from the existing output to make it work with WebCodecs?

[gliese1337]

The timestamp can be computed from the pts (Presentation Time Stamp) or frame_num properties of a Packet object. The chunk type, however, is not encoded in the MPEG-TS container format; getting that information in-band would require partial decoding of the actual video data packets.

There is code in this repository to do a basic level of that, to the extent required for MP4 remuxing (see src/videoData.ts), but it's not cleaned up for publishing yet. An approximation of the chunk type can be computed by looking for the first nalu in a packet with a nal_unit_type of either 1 or 5; a 5 is definitely a key frame, while a 1 might be a delta frame.

If you want to be absolutely certain and don't want to have to do more detailed parsing of the actual video data stream, and you have control over the stream source, it's generally a good idea to just segment your TS stream such that each file starts with a key frame. Otherwise, you'd just have to provide that information out-of-band somehow.

[animeshk874]

Ah okay, understood. Thanks for the quick reply. :)