Cannot get video streaming and read by OpenCV (pyrtmp fails when other RTMP servers succeed)

[petered]

Context: I launch the RTMP server with: python -m pyrtmp.rtmp

I then start streaming a video in terminal with ffmpeg -stream_loop -1 -i /Users/peter/drone/e2_dual/raw/dji_2023-02-26_15-12-52_0167.mov -b:v 50M -f flv rtmp://127.0.0.1/live

And attempt to display the stream with opencv.

Problem 1: The client won't start streaming (solved) This happened because message.publishing_name in the isinstance(message, NSPublish) block of rtmp.py was "", and is fixed by just changing that line to flv = FLVFile(os.path.join(tempfile.gettempdir(), message.publishing_name or "file.flv"))

With that fix, the video starts streaming from terminal, showing

Output #0, flv, to 'rtmp://127.0.0.1/live':
...
frame= 5727 fps= 67 q=2.0 size= 1022955kB time=00:03:11.09 bitrate=43853.7kbits/s speed=2.23x 

and I see lots of debug messages from the server like:

DEBUG:__main__:Receiving VideoMessage(6,1,424424,936425,9,1,936425) 6

So that's good, something's getting through.

Problem 2: OpenCV cannot read the stream (not solved)

This script plots the livestream - and works fine when I use another RTMP server like "Local RTMP Server" (Mac) or "MonaServer" (Windows).

import cv2, time, random

def demo_view_rtmp_stream(stream_url: str):
    print(f"Attempting to connect to stream {stream_url}")
    cap = cv2.VideoCapture(stream_url)
    while True:
        ret, frame = cap.read()
        if not ret:
            print("No frame received.  Skipping.")
            time.sleep(0.5)
            continue
        print(f"Got frame of size {frame.shape}")  # NEVER GETS HERE WHEN USING pyrtmp as server
        cv2.imshow("RTMP Stream", frame)
        cv2.waitKey(1)

if __name__ == '__main__':
    demo_view_rtmp_stream('rtmp://127.0.0.1/live')

If I run this script, and the terminal, when serving with "Local RTMP Server", it works fine.

But, when serving with pyrtmp, it never gets frames, and outputs:

Attempting to connect to stream rtmp://127.0.0.1/live
OpenCV: Couldn't read video stream from file "rtmp://127.0.0.1/live"
[ERROR:0@30.136] global /Users/xperience/actions-runner/_work/opencv-python/opencv-python/opencv/modules/videoio/src/cap.cpp (166) open VIDEOIO(CV_IMAGES): raised OpenCV exception:
OpenCV(4.6.0) /Users/xperience/actions-runner/_work/opencv-python/opencv-python/opencv/modules/videoio/src/cap_images.cpp:253: error: (-5:Bad argument) CAP_IMAGES: can't find starting number (in the name of file): rtmp://127.0.0.1/live in function 'icvExtractPattern'
No frame received.  Skipping.
No frame received.  Skipping.
...

What needs to be done to rtmp.py to make it behave correctly? I assume it somehow needs to pass this CAP_IMAGES into the stream, but I'm not sure how.

[LockedThread]

I wish I could use this too

[Eittipat]

Now PyRTMP only supports video upstream.
Forwarding the video to OpenCV or anything can be done directly by writing an adapter to convert video chunks and feed to OpenCV (or something else). Please look at https://pyav.org/

[LockedThread]

Now PyRTMP only supports video upstream.

Forwarding the video to OpenCV or anything can be done directly by writing an adapter to convert video chunks and feed to OpenCV (or something else). Please look at https://pyav.org/

Thank you for the update.

[bohdankornienko]

Now PyRTMP only supports video upstream. Forwarding the video to OpenCV or anything can be done directly by writing an adapter to convert video chunks and feed to OpenCV (or something else). Please look at https://pyav.org/

I wanted to clarify. By adapter, do you mean something like FLVFileWriter?

class FLVFileWriter:
    def __init__(self, output: str) -> None:
        self.buffer = open(output, "wb")
        self.writer = FLVWriter()
        self.buffer.write(self.writer.write_header())
        super().__init__()

    def write(self, timestamp: int, payload: bytes, media_type: FLVMediaType):
        self.buffer.write(self.writer.write(timestamp, payload, media_type))
        self.buffer.flush()

    def close(self):
        self.buffer.close()

Do I get it right that we need to forward payload to PyAV encoder?