Error When Muxing H.265 Stream to MP4 Using FFmpeg: Timestamps and PTS Issues

[Choiuijin1125]

Hello,

I'm currently working on a video processing project that involves a specific workflow where I decode a video, process the frames through a vision AI model, and then encode the video again. My workflow is as follows:

Decode video -> Convert frame to tensor -> Process frame with a Vision AI model -> Convert tensor back to a surface -> Encode the video

After encoding the video, I obtain a raw H.265 Annex B stream. The final code snippet for the encoding process looks like this:

while True:
    success = nvEnc.FlushSinglePacket(encFrame)
    if success:
        byteArray = bytearray(encFrame)
        dstFile.write(byteArray)
    else:
        break

To mux the raw stream into an MP4 container, I'm using the following FFmpeg command:

ffmpeg -r 29.97 -i input.h265 -vcodec copy output.mp4

However, this results in the following error messages, indicating issues with unset timestamps and missing PTS values:

Timestamps are unset in a packet for stream 0. This is deprecated and will stop working in the future. Fix your code to set the timestamps properly.
[mp4 @ 0x56354b9b5a40] pts has no value

I'm looking for guidance on how to resolve these errors and successfully mux H.265 stream into an MP4 container. Any advice on how to adjust my workflow or the FFmpeg command to fix the timestamp and PTS issues would be greatly appreciated.

[RomanArzumanyan]

Hi @Choiuijin1125

You see those ffmpeg warnings because video track produced by VALI doesn't store PTS and DTS information. Reason for that is PTS and DTS are not part of video coding standards. Hence VALI has no place to insert them into compressed video frame which PyNvEncoder returns. These timestamps only exist on the container level and are to be inserted by muxer.

One way to avoid the ffmpeg warnings is to pass genpts option. It will tell ffmpeg to generate timestamps based on FPS:

ffmpeg -fflags +genpts -r 29.97 -i input.h266 -c:v copy -y output.mp4

P. S. This is important question which is often asked so I'm moving it to Discussion to be permanently visible.