search

PyAV-Org / PyAV

Pythonic bindings for FFmpeg's libraries.
https://pyav.basswood-io.com/
BSD 3-Clause "New" or "Revised" License
2.47k stars 360 forks source link

Unable to set frame rate #1411

Closed tyler-rt closed 4 months ago

tyler-rt commented 4 months ago

Overview

When using a webcam, pyav fails to set frame rate.

Expected behavior

Frame rate should be changeable, as in this ffmpeg script which on my computer works at 60fps:

Actual behavior

Frame rate is not set. cc https://github.com/PyAV-Org/PyAV/issues/1005

Investigation

I made a script with ffmpeg showing that I can change the FPS for my webcam from the default of 30fps to say 60fps or 15fps:

import cv2
import subprocess
import numpy as np
import time

# FFmpeg command
ffmpeg_command = [
    'ffmpeg',
    '-f', 'v4l2',
    '-framerate', '60',
    '-video_size', '1920x1080',
    '-i', '/dev/video4',
    '-f', 'rawvideo',
    '-pix_fmt', 'rgb24',
    'pipe:1'
]

# Start the FFmpeg process and suppress its output
process = subprocess.Popen(ffmpeg_command, stdout=subprocess.PIPE, stderr=subprocess.PIPE)

# Set up OpenCV window
cv2.namedWindow('Webcam', cv2.WINDOW_NORMAL)
cv2.resizeWindow('Webcam', 1920, 1080)

frame_count = 0
start_time = time.time()

while True:
    # Read raw video frame from stdout
    raw_frame = process.stdout.read(1920 * 1080 * 3)

    if len(raw_frame) != 1920 * 1080 * 3:
        break

    # Convert raw frame to numpy array
    frame = np.frombuffer(raw_frame, np.uint8).reshape((1080, 1920, 3))

    # Display the frame using OpenCV
    cv2.imshow('Webcam', frame)

    frame_count += 1

    # Calculate and print FPS every second
    elapsed_time = time.time() - start_time
    if elapsed_time >= 1.0:
        fps = frame_count / elapsed_time
        print(f"Average FPS: {fps:.2f}")
        frame_count = 0
        start_time = time.time()

    # Exit on ESC key
    if cv2.waitKey(1) & 0xFF == 27:
        break

# Clean up
process.stdout.close()
process.wait()
process.terminate()
cv2.destroyAllWindows()

Reproduction

PyAV is unable to change frame rate:

import av
import cv2
import time
from fractions import Fraction

def open_usb_webcam(max_devices=10):
    "Open VideoCapture with the last valid device (e.g. USB webcam) on the system."
    for device_index in range(max_devices, 0, -1):
        try:
            input_device = f"/dev/video{device_index}"
            # container = av.open(input_device)
            container = av.open(input_device)
            stream = container.streams.video[0]
            return container, stream
        except (av.AVError, IndexError):
            continue
    raise ValueError("No valid video capture devices found.")

# Open the webcam
container = av.open("/dev/video0")
stream = container.streams.video[0]

# Configure the stream for the specified settings
# Tested that we can change the stream width and height
stream.width = 1920
stream.height = 1080
# stream.width = 800
# stream.height = 600
# pix_fmt doesn't appear to matter for setting frame rate
stream.pix_fmt = 'yuv422p'
# however, we are unable to set the frame rate
stream.rate = Fraction(1,60)
# stream.framerate = Fraction(1,15)
# stream.rate = "15"

# Create OpenCV window
cv2.namedWindow('Webcam', cv2.WINDOW_NORMAL)
cv2.resizeWindow('Webcam', 1920, 1080)

# Start time
start_time = time.time()
frame_count = 0

duration = 3
for packet in container.demux(stream):
    current_time = time.time()
    if current_time - start_time >= duration:
        break

    for frame in packet.decode():
        # Convert frame to OpenCV format (BGR)
        img = frame.to_ndarray(format='bgr24')

        # # Display the frame
        # commenting out display has no impact on fps
        cv2.imshow('Webcam', img)
        frame_count += 1

        if cv2.waitKey(1) & 0xFF == ord('q'):
            break
        pass

    if current_time - start_time >= duration:
        break

# Calculate and print the average FPS
elapsed_time = time.time() - start_time
average_fps = frame_count / elapsed_time
print(f"Average FPS: {average_fps:.2f}") # 30.00 FPS ???
print(img.shape)
# Release resources
cv2.destroyAllWindows()
container.close()

Versions

Research

I have done the following:

Additional context

WyattBlue commented 4 months ago 
stream.rate = Fraction(1,60)

is deprecated, and is removed in 12.1.0, released today.

WyattBlue commented 4 months ago

anyway, you shouldn't be able to set the stream frame rate? (How would you specify variable frame rates?)

You're looking for either:

tyler-rt commented 4 months ago

@WyattBlue indeed I want to set the device frame rate of the webcam so it acquires at 15 or 60 fps. Is this supported by PyAV currently? In ffmpeg this is done with -framerate 60