Typing error

[duburcqa]

Overview

mypy is throwing an error with the latest release of pyav (12.0) which is now fully typed as far as a now.

Expected behavior

By looking at the unit tests, I am quite sure that this usecase is supported by design and therefore it should not raise any error.

Actual behavior

Here is the exact trace:

error: Argument 1 to "memoryview" has
incompatible type "VideoPlane"; expected "Buffer"  [arg-type]
                        memoryview(frame.planes[0])[:] = buffer

Investigation

I think the problem is pretty well identified already, there is a typing issue because VideoPlane is not declared as implmenting Python Buffer Interface from type checker perspective.

Reproduction

I could write one if necessary but it sounds pretty straightforward already.

Versions

• OS: macOS 14.3
• PyAV runtime:

  {{ Complete output of `python -m av --version`. If this command won't run, you are likely dealing with the build issue and should use the appropriate template. }}

• PyAV build:

  {{ Complete output of `python setup.py config --verbose`. }}

• FFmpeg:

  {{ Complete output of `ffmpeg -version` }}

Research

I have done the following:

• [x] Checked the PyAV documentation
• [ ] Searched on Google
• [ ] Searched on Stack Overflow
• [ ] Looked through old GitHub issues
• [ ] Asked on PyAV Gitter
• [ ] ... and waited 72 hours for a response.

[maxstrobel]

Maybe I can hook me in here.

I have a use case, which is pretty close to the Generating Video example from the cookbook. It throws also errors, but the example runs without any issue.

Reproduce

mypy gen_video.py
gen_video.py:12: error: "Stream" has no attribute "width"  [attr-defined]
gen_video.py:13: error: "Stream" has no attribute "height"  [attr-defined]
gen_video.py:14: error: "Stream" has no attribute "pix_fmt"  [attr-defined]
gen_video.py:24: error: Argument 1 to "from_ndarray" of "VideoFrame" has incompatible type "ndarray[Any, dtype[floating[_64Bit]]]"; expected "ndarray[Any, dtype[unsignedinteger[_8Bit]]] | ndarray[Any, dtype[unsignedinteger[_16Bit]]] | ndarray[Any, dtype[floating[_32Bit]]]"  [arg-type]
gen_video.py:25: error: "Stream" has no attribute "encode"  [attr-defined]
gen_video.py:29: error: "Stream" has no attribute "encode"  [attr-defined]
Found 6 errors in 1 file (checked 1 source file)

gen_video.py:

import numpy as np

import av

duration = 4
fps = 24
total_frames = duration * fps

container = av.open("test.mp4", mode="w")

stream = container.add_stream("mpeg4", rate=fps)
stream.width = 480
stream.height = 320
stream.pix_fmt = "yuv420p"

for frame_i in range(total_frames):
    img = np.empty((480, 320, 3))
    img[:, :, 0] = 0.5 + 0.5 * np.sin(2 * np.pi * (0 / 3 + frame_i / total_frames))
    img[:, :, 1] = 0.5 + 0.5 * np.sin(2 * np.pi * (1 / 3 + frame_i / total_frames))
    img[:, :, 2] = 0.5 + 0.5 * np.sin(2 * np.pi * (2 / 3 + frame_i / total_frames))

    img = np.round(255 * img).astype(np.uint8)

    frame = av.VideoFrame.from_ndarray(img, format="rgb24")
    for packet in stream.encode(frame):
        container.mux(packet)

# Flush stream
for packet in stream.encode():
    container.mux(packet)

# Close the file
container.close()

My environment:

OS: Windows 11

python --version                        
Python 3.11.7

pip list
Package                       Version                               Editable project location
----------------------------- ------------------------------------- -------------------------
...
av                            12.0.0
...
mypy                          1.4.1
mypy-extensions               1.0.0
...

[WyattBlue]

VideoPlane is not declared as implementing Python Buffer Interface from type checker perspective. True, we should fix this.

@maxstrobel most of the errors would be fixed if you casted stream to VideoStream.

I'm going to close this because I want typing issues to be precise.

[duburcqa]

What do you mean by "precise". mypy errors are as precise as it can be. It clearly shows that the typing annotations of PyAV are wrong and it is very clear why they are wrong. It cannot be any more "precise" than that. By if you don't care that's fine for me, i still have the option to disable PyAV typing completely anyway.

[maxstrobel]

@WyattBlue - I casted the stream to VideoStream as you suggested; still there's an issue, which is hopefully precise enough.

Reproduce:

mypy gen_video.py  
gen_video.py:28: error: Argument 1 to "mux" of "OutputContainer" has incompatible type "Packet"; expected "Sequence[Packet]"  [arg-type]
gen_video.py:32: error: Argument 1 to "mux" of "OutputContainer" has incompatible type "Packet"; expected "Sequence[Packet]"  [arg-type]
Found 2 errors in 1 file (checked 1 source file)

gen_video.py:

from typing import cast

import av
import numpy as np
from av.video.stream import VideoStream

duration = 4
fps = 24
total_frames = duration * fps

container = av.open("test.mp4", mode="w")

stream = cast(VideoStream, container.add_stream("mpeg4", rate=fps))  # noqa: F821
stream.width = 480
stream.height = 320
stream.pix_fmt = "yuv420p"

for frame_i in range(total_frames):
    _img = np.empty((480, 320, 3))
    _img[:, :, 0] = 0.5 + 0.5 * np.sin(2 * np.pi * (0 / 3 + frame_i / total_frames))
    _img[:, :, 1] = 0.5 + 0.5 * np.sin(2 * np.pi * (1 / 3 + frame_i / total_frames))
    _img[:, :, 2] = 0.5 + 0.5 * np.sin(2 * np.pi * (2 / 3 + frame_i / total_frames))

    img = np.round(255 * _img).astype(np.uint8)

    frame = av.VideoFrame.from_ndarray(img, format="rgb24")
    for packet in stream.encode(frame):
        container.mux(packet)

# Flush stream
for packet in stream.encode():     # <------- packet is a "single Packet"
    container.mux(packet)

# Close the file
container.close()

[WyattBlue]

@maxstrobel Try using the main branch's type stubs.

[maxstrobel]

@WyattBlue Thanks!

Is there a plan for a bugfix release to cover those things, or will be the next version the scheduled v13.0.0?