search

PyAV-Org / PyAV

Pythonic bindings for FFmpeg's libraries.
https://pyav.basswood-io.com/
BSD 3-Clause "New" or "Revised" License
2.44k stars 359 forks source link

error: OpenCV #1326

Closed sugizo closed 6 months ago

sugizo commented 6 months ago

IMPORTANT: Be sure to replace all template sections {{ like this }} or your issue may be discarded.

Overview

video output can not work with opencv

Expected behavior

video output can get image filter using opencv

Actual behavior

an error occured

Traceback:

error                                     Traceback (most recent call last)
[<ipython-input-4-4113c5a3d8b1>](https://localhost:8080/#) in <cell line: 20>()
     26     packet.stream = out_stream
     27 
---> 28     gray = cv2.cvtColor(packet, cv2.COLOR_BGR2GRAY)
     29     #gray = cv2.cvtColor(out_stream, cv2.COLOR_BGR2GRAY) # not work
     30 

error: OpenCV(4.8.0) :-1: error: (-5:Bad argument) in function 'cvtColor'
> Overload resolution failed:
>  - src is not a numpy array, neither a scalar
>  - Expected Ptr<cv::UMat> for argument 'src'

Investigation

    gray = cv2.cvtColor(packet, cv2.COLOR_BGR2GRAY) # not work
    #gray = cv2.cvtColor(out_stream, cv2.COLOR_BGR2GRAY) # not work

Reproduction

import av
import av.datasets
import IPython
import os
import subprocess
import cv2

temp_file = "temp.mp4"
output_file = "merge_audio_ffmpeg.mp4"
audio_file = 'sheep.mp3'

input_ = av.open(av.datasets.curated("pexels/time-lapse-video-of-night-sky-857195.mp4") )
output = av.open(temp_file, "w")

# Make an output stream using the input as a template. This copies the stream
# setup from one to the other.
in_stream = input_.streams.video[0]
out_stream = output.add_stream(template=in_stream)

for packet in input_.demux(in_stream):
    # We need to skip the "flushing" packets that `demux` generates.
    if packet.dts is None:
        continue

    # We need to assign the packet to the new stream.
    packet.stream = out_stream

    gray = cv2.cvtColor(packet, cv2.COLOR_BGR2GRAY)
    #gray = cv2.cvtColor(out_stream, cv2.COLOR_BGR2GRAY) # not work

    output.mux(gray)

input_.close()
output.close()

if not os.path.exists(output_file):
    subprocess.check_call(
        [
            "ffmpeg",
            "-i",
            temp_file,
            "-i",
            audio_file,
            "-vcodec",
            "libx264",
            "-acodec",
            "aac",
            "-map",
            "0:v:0",
            "-map",
            "1:a:0",
            "-shortest",
            output_file,
        ]
    )

os.remove(temp_file)

IPython.display.Video(output_file, embed = True)

Versions

Research

I have done the following:

Additional context

WyattBlue commented 6 months ago

The error occurs when an opencv function is called, so this is outside my area of expertise. I do not know the type signature of cv2.cvtColor or what gray = cv2.cvtColor(packet, cv2.COLOR_BGR2GRAY) is supposed to even accomplish. I recommend asking people who work with opencv-python.