frame pts is zero - Githubissues

jhnbyrn commented 5 years ago

I'm having a strange issue with frames I read from an av.container. It doesn't seem to happen with ffprobe on the command line. The issue is that the frame.pts and frame.time values are zero for more than half of the frames. I get bursts of 20-40 frames with proper timestamps, then at least that number of zeros, often more. Here's a script that demonstrates it:

import av

if __name__ == '__main__':
    container = av.open(file='/dev/video0',
                        format='v4l2',
                        mode='r',
                        options={'video_size':'320x240'})

    while(True):
        frame = next(container.decode())
        print(frame.pts)

Running this, I get output like this (edited for brevity):

aiortc_py3) pi@raspberrypi:~/projects/av_issue $ python frames.py 
325219215
325352522
325385849
325419176
...
[_CONTINUES LIKE THIS FOR 20 OR SO FRAMES_]
...
326418981
0
0
0
...
[_NOTHING BUT ZEROS FOR 40 FRAMES OR SO_]
...
0
0
327985339
328018666
328051993
328085320
328118647
328151973
328185302
328218628
328251954

This pattern seems to repeat itself indefinitely. There's some variation in the amount of good vs bad frames, but not much.

I also tried saving the frames as images and checked them - they look fine, so it seems to be just the times that are a problem.

I know pyav gets the frame data from ffmpeg, but when I run ffprobe to check the frames like this:

pi@raspberrypi:~/projects/av_issue $ /usr/bin/ffprobe -i /dev/video0 -show_frames | grep pkt_pts=

...then all the timestamps look fine - no zeros.

Could there be something in the way I'm calling open(...) that's causing an issue? I'd like to see what other "options" are available, but I can't find any documentation for this. Can you point me towards anything?

I'm using a raspberry pi with the bcm2835-v4l2 camera driver, and pyav 6.1.2.

ffmpeg version is 3.2.12-1. I've built a newer one from the latest source, but don't know how to make pyyav use that one instead. But since this version seems to get the frames correctly on the command line, I'm not sure if that's the best direction to look.

Can you give me any suggestions for further debugging? Thanks!

jhnbyrn commented 5 years ago

I was able to get it working with the latest ffmpeg, after building from source. I wasn't able to reproduce the issue using ffmpeg alone, but the latest version doesn't seem to have the issue with PyAV.

jlaine commented 5 years ago

Wow, that's an unexpected conclusion. For the record could you state what version had the issue / what version worked? Thanks!

jhnbyrn commented 5 years ago

Wow, that's an unexpected conclusion. For the record could you state what version had the issue / what version worked? Thanks!

I had the issue with PyAV 6.1.2 and ffmpeg 3.2.12-1~deb9u1+rpt1

The issue is not present with the latest PyAV source and a recent ffmpeg source build. The version numbers I'm getting from my own builds are PyAV 6.1.3 and ffmpeg N-93320-g5ab44ff20c.

So maybe it was a change in PyAV...I hadn't really thought about that - I only built it because I assumed I had to do that to get it to use the new ffmpeg build.

It might be relevant that the configuration options are different. The ffmpeg version that had the issue is the raspbian package, and has a long list of config options. I would have liked to keep them exactly the same for my build, but I was getting errors - apparently some of them aren't supported any longer - so I did my build with a very minimal set of options.

Let me know if there's any other info that would be useful.

jlaine commented 5 years ago

Well it would probably help to know if the following combo works:

latest PyAV from pypi
latest official ffmpeg release (4.1.x)

jhnbyrn commented 5 years ago

Well it would probably help to know if the following combo works:
* latest PyAV from pypi

* latest official ffmpeg release (4.1.x)

*removed because it was probably a bad test*

EDIT: Sorry, I might have to take that back. I don't think I fully understand how PyAV and ffmpeg are linked. I now think the PyAV from the pypi is using the older version of ffmpeg, which is installed in a different place. I'll post back again when I have figured out how to test it properly. Sorry for the confusion.

PyAV-Org / PyAV

frame pts is zero #493