search

PyAV-Org / PyAV

Pythonic bindings for FFmpeg's libraries.
https://pyav.basswood-io.com/
BSD 3-Clause "New" or "Revised" License
2.5k stars 364 forks source link

Inconsistent data packet sizes #1467

Closed mattjdnv closed 3 months ago

mattjdnv commented 3 months ago

Overview

I am reading and parsing a mpeg ts file with video and a KLV metadata stream using a Rocky Linux 9 docker container with pyAV,

The missing 5 bytes make it difficult to parse the KLV stream.

Expected behavior

Packet size should be the same between versions. The following Python code gives different packet sizes for pyAV v11 vs pyAVv12+ NOTE: If there is a better way to do this, I'd like to do it.

container = av.open(invideo)
for video_frame, klv_packet in zip(container.decode(video=0), container.demux(data=0)):
    print (klv_packet)
    print (video_frame.to_image())
    print ("---")

Actual behavior

Output from pyAV v11.0.0

dev@9c3a4c519284 tst3]$ ./dumpVid.sh 
pyAV: 11.0.0
Processing: vis1_11.0.0
start time for stream 1 is not set in estimate_timings_from_pts
<PIL.Image.Image image mode=RGB size=1920x1080 at 0x7151779DAD00>
<av.Packet of #1, dts=None, pts=None; 255 bytes at 0x71517780f4a0>
---
<PIL.Image.Image image mode=RGB size=1920x1080 at 0x7151779DAF10>
<av.Packet of #1, dts=None, pts=None; 255 bytes at 0x715177814e50>
---
<PIL.Image.Image image mode=RGB size=1920x1080 at 0x7151779DAD00>
<av.Packet of #1, dts=None, pts=None; 255 bytes at 0x71517771d3b0>
---

Output from pyAVv12.3.0

[dev@098678f8a38c tst3]$ ./dumpVid.sh 
pyAV: 12.3.0
Processing: vis1_12.3.0
<PIL.Image.Image image mode=RGB size=1920x1080 at 0x7293465E3F70>
<av.Packet of #1, dts=None, pts=None; 250 bytes at 0x729346402680>
---
<PIL.Image.Image image mode=RGB size=1920x1080 at 0x7293465E3F70>
<av.Packet of #1, dts=None, pts=None; 250 bytes at 0x729346408ef0>
---
<PIL.Image.Image image mode=RGB size=1920x1080 at 0x7293465E3F70>
<av.Packet of #1, dts=None, pts=None; 250 bytes at 0x7293462f6360>
---

Traceback:

{{ Include complete tracebacks if there are any exceptions. }}

Investigation

Tested running the python code in docker containers with each of the pyAV releases listed in the Changelog.

Reproduction

I'm not able to send a video sample.

Versions

pyAV v12

PyAV v12.3.0
library configuration: --disable-static --enable-shared --libdir=/tmp/vendor/lib --prefix=/tmp/vendor --disable-alsa --disable-doc --disable-libtheora --disable-mediafoundation --enable-fontconfig --enable-gmp --enable-gnutls --enable-libaom --enable-libbluray --enable-libdav1d --enable-libfreetype --enable-libmp3lame --enable-libopencore-amrnb --enable-libopencore-amrwb --enable-libopenjpeg --enable-libopus --enable-libspeex --enable-libtwolame --enable-libvorbis --enable-libvpx --enable-libwebp --enable-libxcb --enable-libxml2 --enable-lzma --enable-zlib --enable-version3 --enable-libx264 --disable-libopenh264 --enable-libx265 --enable-gpl
library license: GPL version 3 or later
libavcodec     60. 31.102
libavdevice    60.  3.100
libavfilter     9. 12.100
libavformat    60. 16.100
libavutil      58. 29.100
libswresample   4. 12.100
libswscale      7.  5.100

Research

I have done the following:

Additional context

{{ Add any other context about the problem here. }}

moonsikpark commented 3 months ago

Can you provide a sample file with minimal reproducible example? If not able, please perform bisection to find the offending commit.

WyattBlue commented 3 months ago

"The missing 5 bytes make it difficult to parse the KLV stream." This is a crazy statement. How can parsing bytes less than 255 possibly be difficult? Maybe you meant to say possible/impossible. That at least makes sense to me.

Any, I swear I someone complained before that the len of bytes(packet) was different between major versions. Which yeah, that's why they're different major versions. When FFmpeg changes, so does PyAV.

WyattBlue commented 3 months ago

Found it https://github.com/PyAV-Org/PyAV/issues/1376. If you can send a patch that can address this, go for it. Otherwise, don't complain in the issue tracker. I won't do this work for you.

mattjdnv commented 3 months ago

"The missing 5 bytes make it difficult to parse the KLV stream." This is a crazy statement. How can parsing bytes less than 255 possibly be difficult? Maybe you meant to say possible/impossible. That at least makes sense to me.

Any, I swear I someone complained before that the len of bytes(packet) was different between major versions. Which yeah, that's why they're different major versions. When FFmpeg changes, so does PyAV.

I will have a go at patching it.

The reason I said "difficult" is each KLV packet has a set of identification bytes. If you can't read it you have trouble figuring out what the packet is.
From the MISB ST 1601.2 specification:

6 Geo-Registration Local Set
The Geo-Registration Local Set is a KLV Local Set whose items provide information specific to 
a geo-registration algorithm, such as name and version, and additional geo-registration output, 
such as correspondence points. All KLV metadata needs to conform to MISB ST 0107 [5] and 
SMPTE 336 [6] encoding rules.

6.1 Conventions
Requirement(s)
ST 1601-01 All metadata shall be expressed in accordance with MISB ST 0107.
ST 1601-02 Formatting of Geo-Registration metadata shall be compliant with SMPTE ST 336
encoding rules for Universal Labels and Local Sets.

6.2 Geo-Registration Local Set
The Geo-Registration Local Set 16-Byte Universal Label registered in MISB ST 0807 [7] is:
06.0E.2B.34.02.0B.01.01.0E.01.03.03.01.00.00.00 (CRC 39238)

The "missing bytes" are the first 5 bytes in that label.