How do I remux to BytesIO?

[simunovic-antonio]

Overview

I'm trying to remux a RTSP H264 stream from IP camera to fragmented MP4 stream to use it in a live streaming WebSocket solution. On the client side is the MediaSource object. Having trouble getting the bytes result from the output buffer, where getvalue() call end with an empty bytes object for all non-keyframe packets.

Here is the code:

import io
import av

input_container = av.open("rtsp://CAMERA_IP_ADDRESS:554/Streaming/Channels/101", "r")
input_stream = input_container.streams.video[0]

output_buffer = io.BytesIO()
output_container = av.open(output_buffer, 'w', format="mp4", options={"strict":"-1", "movflags":"empty_moov+omit_tfhd_offset+frag_keyframe+default_base_moof"})
output_stream = output_container.add_stream(template=input_stream)

first_packet = True
for packet in input_container.demux(input_stream):
    if first_packet:
        packet.dts = 0
        packet.pts = 0
        first_packet = False
    packet.stream = output_stream
    output_container.mux_one(packet)
    result = output_buffer.getvalue()
    if result:
        print("YES", packet.is_keyframe)
        output_buffer.truncate(0)
        output_buffer.seek(0)
    else:
        print("NO", packet.is_keyframe)

Here is the output:

YES True
NO False
NO False
...
NO False
NO False
YES True
NO False

Expected behavior

I expect that every call to getvalue() returns a non-empty bytes object.

Actual behavior

Only after muxing keyframe packet I will get non-empty result in call to BytesIO.getvalue().

Research

I have seen a perhaps similar question in Issue #524 with no answers.

I have tried building PyAv with change in the mux_one method of the OutputContainer class. Change was in using av_write_frame function instead of av_interleaved_write_frame. My understanding was that the later one caches packets due to the need to interleave packets from multiple streams, but the former one does not. But the end result is the same, get_value returns non-empty result only on key-frame packets.

[jlaine]

No idea here, and since you're using a modified version of PyAV this feels out of scope..

[simunovic-antonio]

Using the official build produces the same results.