854 * 480 resolution causes "pixel buffer size mismatch" on macOS

[agilebean]

• Operating system: MacOS Venture 13.5
• Python version: 3.8
• pyvirtualcam version: 0.10.2
• Virtual camera (OBS, v4l2loopback, UnityCapture): OBS
• Virtual camera version: 29.1.3

Describe the bug ... Running a python script sending frames from a video which has worked perfectly one year ago. The same script keeps on throwing the error

pixel buffer size mismatch

Notes:

1. This happened no matter which color conversion was done (*any combination).
2. cv2.imshow() works perfectly.
3. OBS Studio works perfectly, i.e. sends the same video to a virtual cam which can be seen in Google Meet or Zoom.

To Reproduce Trying to run from this repository the example script webcam_filter.py

import logging
import argparse
import pyvirtualcam
from pyvirtualcam import PixelFormat
import cv2

parser = argparse.ArgumentParser()
parser.add_argument("--video_path", help="path to input video file", default="video/dicaprio1-silent_rgb.mp4")
parser.add_argument("--fps", action="store_true", help="output fps every second")
parser.add_argument("--device", help="virtual camera device, e.g. /dev/video0 (optional)")
args = parser.parse_args()

print(args.video_path)

video = cv2.VideoCapture(args.video_path)
if not video.isOpened():
    raise ValueError("error opening video")
length = int(video.get(cv2.CAP_PROP_FRAME_COUNT))
width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))
fps = video.get(cv2.CAP_PROP_FPS)

print(f"width {width} height{height}")

with pyvirtualcam.Camera(width=width, height=height, fps=fps, fmt=PixelFormat.BGR,
                         device=args.device) as cam:

    print(f'Virtual cam started: {cam.device} ({cam.width}x{cam.height} @ {cam.fps}fps)')
    # Set the resolution of the virtual camera

    count = 0
    while True:
        # Restart video on last frame.
        if count == length:
            count = 0
            video.set(cv2.CAP_PROP_POS_FRAMES, 0)

        # Read video frame.
        ret, frame = video.read()
        if not ret:
            raise RuntimeError('Error fetching frame')

        cv2.imshow('frame: ', frame)
        # frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR)
        # frame = cv2.cvtColor(frame, cv2.COLOR_RGB2BGR) # UVYV

        # Send to virtual cam.
        cam.send(frame)

        # Wait until it's time for the next frame
        cam.sleep_until_next_frame()

        count += 1

[letmaik]

What is the width and height of that video?

[agilebean]

It is 854 * 480. But I get both measurements by video.get() as in

width = int(video.get(cv2.CAP_PROP_FRAME_WIDTH))
height = int(video.get(cv2.CAP_PROP_FRAME_HEIGHT))

so it shouldn't matter, should it?

[letmaik]

Can you try with a different video that is 1280x720?

[agilebean]

@letmaik OMG you were right! You are a genius!!!

The 1280x720 is correctly received in Zoom! But why??? I initialize pyvirtualcam with the video dimensions, not hardcoded.

What do I do wrong here:

 def _process_video(self, video_path):
        logging.info(f"************************************************************")
        logging.info(f"VideoCapture() from file >> : {video_path}")
        video_capture = cv2.VideoCapture(video_path)
        if not video_capture.isOpened():
            raise ValueError("Error opening video")
        audio_stream = MediaPlayer(video_path, ff_opts={})

        self.width = int(video_capture.get(cv2.CAP_PROP_FRAME_WIDTH))
        self.height = int(video_capture.get(cv2.CAP_PROP_FRAME_HEIGHT))
        self.fps = video_capture.get(cv2.CAP_PROP_FPS)

        self.virtualcam = pyvirtualcam.Camera(
            width=self.width,
            height=self.height,
            fps=self.fps,
            device="OBS Virtual Camera",
            fmt=pyvirtualcam.PixelFormat.BGR
        )
        print(f"CAM: {self.virtualcam}")
        # if not self.audio_stream.isOpened():
        #     raise ValueError("Error opening audio")
        logging.info(f"_process_video() : {str(type(video_capture))}{str(type(audio_stream))}")

and then just sending it:

        def send_frame_to_virtual_cam(self):
            if self.os in [LINUX, LINUX2]:

                try:
                    # frame = cv2.cvtColor(frame, cv2.COLOR_YUV2BGR)
                    self.virtualcam.schedule_frame(frame)
                except Exception as e:
                    logging.error(f"An error occurred: {e}")
                except KeyboardInterrupt:
                    self.virtualcam.close()
                    video_capture.release()

            elif self.os in [MAC, WINDOWS]:
                try:
                    logging.info("Before send")

                    self.virtualcam.send(frame)
                    logging.info("After send")

                    if hasattr(self.virtualcam, '_last_frame_t') and self.virtualcam._last_frame_t is not None:
                        logging.info(f"_last_frame_t: {self.virtualcam._last_frame_t}")
                        self.virtualcam.sleep_until_next_frame()
                except Exception as e:
                    logging.error(f"An error occurred: {e}")
                except KeyboardInterrupt:
                    self.virtualcam.close()
                    video_capture.release()

NOTE: Sending a frame with different dimensions prints the output (however doesn't throw an error):

pixel buffer size mismatch

NOTE 2: Strangely, I'm getting the warning

  import cv2

but I don't see where I import numpy:

import grpc
from api import broadcast_pb2
from api import broadcast_pb2_grpc
from google.protobuf.empty_pb2 import Empty

import logging
import colorlog
import sys
import argparse
import asyncio
import threading
import time
import cv2
import pyvirtualcam
from ffpyplayer.player import MediaPlayer

[letmaik]

I can reproduce the problem. I tried to relax the condition below as I thought the buffer may be bigger sometimes and while that produces an image, it is distorted, so not the right fix: https://github.com/letmaik/pyvirtualcam/blob/44e8fa279c9b4a94733bab58a3f2119f50b91d7a/pyvirtualcam/native_macos_obs/virtual_output.h#L239-L243

For now, I suggest to stick to the standard 1280x720 resolution and resize your frames in advance. Eventually this should be fixed but I don't have much experience with Macs, so hopefully someone else can help out.

[agilebean]

For now, I suggest to stick to the standard 1280x720 resolution and resize your frames in advance. Eventually this should be fixed but I don't have much experience with Macs, so hopefully someone else can help out.

That's oddly reassuring that I'm not crazy or didn't do an obvious mistake. Thank you very much indeed. On the other hand, that means that setting the height and width attributes in pyVirtualCam.Camera() doesn't work! I guess you're hinting that this is more likely MacOS specific, right?

[GermanWarez]

"Video width must be divisible by 8" is quite a few year old. But I'm pretty sure the green image is caused by OBS virtual camera plugin encoding the images to a video, and not related to pyvirtualcam.

And as far as I understand the tests done on July 06: The user send frames with different sizes on purpose, and this was detected and reported as 'pixel buffer size mismatch'. -> This not related to the reported bug (green image) and proves that the buffer check works. @agilebean if you could clarify, please.

And if you can record the green video with VLC (or any other tool) and report the used encoder it might help to reproduce the issue independent from OBS. e.g. with ffmpeg.

[agilebean]

@GermanWarez In my script, the width and height were read programmatically. So no, the frames were not send with different sizes on purpose.

However, changing from the width and height attributes (854 480) to 1280 720 did the trick.

Yet, this was the case when the OBS plugin still worked but it stopped working with the newer MacOS (I filed another issue but unfortunatelty hasn't been solved yet).

[GermanWarez]

Thanks for clarification.

Then it's either a bug in cv2 producing different frames sizes (easy to log in python), or a bug in libyuv in virtual_output.h (that is used to convert the frame format). Did you test with UYVY frames, too (cv.COLOR_BGR2YUV_Y422)? Those are not converted in virtual)output.h:

            case libyuv::FOURCC_UYVY:
                out_frame = const_cast<uint8_t*>(frame);

I couldn't find any other python library using libyuv, and can't test pyvirtualcam on macOS 13.

If you could log the frame sizes by cv2 in python, and try UYVU frames it would be possible to pinpoint the error either to cv2 or libyuv. Or both are affected... Printing the frame size:

     height, width, _ = frame.shape
    print(f"Frame Size: {width}x{height}")

ty