abhiTronix / vidgear

A High-performance cross-platform Video Processing Python framework powerpacked with unique trailblazing features :fire:
https://abhitronix.github.io/vidgear
Apache License 2.0
3.37k stars 253 forks source link

How do I display two streams simultaneously? #224

Closed PrashantSaikia closed 3 years ago

PrashantSaikia commented 3 years ago

Let's take the example code here that displays the webcam feed using OpenCV. Say I want two of these feeds to be displayed side by side - one for the webcam stream, and one for streaming another video stream (like a pre-downloaded mp4 video, etc). How would I do that? I'm guessing I'll need to have two async custom frame producers. But I tried it the following way:

web.config["generator"] = my_frame_producer_1
web.config["generator"] = my_frame_producer_2

and one overlaps the other. So, I'm guessing there should be a way to give the location coordinates to display the stream. That should solve the overlap issue. How do I do that?

welcome[bot] commented 3 years ago

Thanks for opening this issue, a maintainer will get back to you shortly!

In the meantime:

abhiTronix commented 3 years ago

Say I want two of these feeds to be displayed side by side - one for the webcam stream, and one for streaming another video stream (like a pre-downloaded mp4 video, etc).

@PrashantSaikia Good question, I was hoping someone to ask this. Thanks to modular stucture of WebGear, the simplest trick here is to run one source generator directly through config["generator"] and other source through different Streaming Response and Route. In this way you can easily create any number of source generators over different routes.

For example in below example, I will run first source "foo1.mp4" from my_frame_producer1 directly through web.config["generator"] = my_frame_producer1 as given in docs. But for second source "foo2.mp4", I'll create a new custom_video_response and assign it to a completely different route i.e. /video2, thereby you can easily access it on different URL(http://localhost:8000/video2) after running this example:

# import necessary libs
import uvicorn, asyncio, cv2
from vidgear.gears.asyncio import WebGear
from vidgear.gears.asyncio.helper import reducer
from starlette.responses import StreamingResponse
from starlette.routing import Route

# initialize WebGear app without any source
web = WebGear(logging=True)

# create your own custom frame producer
async def my_frame_producer1():

    # !!! define your own video source here !!!
    # Open any video stream such as "foo1.mp4"
    stream = cv2.VideoCapture("foo1.mp4")
    # loop over frames
    while True:
        # read frame from provided source
        (grabbed, frame) = stream.read()
        # break if NoneType
        if not grabbed:
            break

        # do something with your OpenCV frame here

        # reducer frames size if you want more performance otherwise comment this line
        frame = await reducer(frame, percentage=30)  # reduce frame by 30%
        # handle JPEG encoding
        encodedImage = cv2.imencode(".jpg", frame)[1].tobytes()
        # yield frame in byte format
        yield (b"--frame\r\nContent-Type:video/jpeg2000\r\n\r\n" + encodedImage + b"\r\n")
        await asyncio.sleep(0.00001)
    # close stream
    stream.release()

# create your own custom frame producer
async def my_frame_producer2():

    # !!! define your own video source here !!!
    # Open any video stream such as "foo2.mp4"
    stream = cv2.VideoCapture("foo2.mp4")
    # loop over frames
    while True:
        # read frame from provided source
        (grabbed, frame) = stream.read()
        # break if NoneType
        if not grabbed:
            break

        # do something with your OpenCV frame here

        # reducer frames size if you want more performance otherwise comment this line
        frame = await reducer(frame, percentage=30)  # reduce frame by 30%
        # handle JPEG encoding
        encodedImage = cv2.imencode(".jpg", frame)[1].tobytes()
        # yield frame in byte format
        yield (b"--frame\r\nContent-Type:video/jpeg2000\r\n\r\n" + encodedImage + b"\r\n")
        await asyncio.sleep(0.00001)
    # close stream
    stream.release()

async def custom_video_response(scope):
    """
    Return a async video streaming response for `my_frame_producer2` generator
    """
    assert scope["type"] in ["http", "https"]
    await asyncio.sleep(0.00001)
    return StreamingResponse(
        my_frame_producer2(),
        media_type="multipart/x-mixed-replace; boundary=frame",
    )

# add your custom frame producer to config
web.config["generator"] = my_frame_producer1

# append new route i.e. new custom route with custom response
web.routes.append(
     Route("/video2", endpoint=custom_video_response)
     )

# run this app on Uvicorn server at address http://localhost:8000/
uvicorn.run(web(), host="localhost", port=8000)

# close app safely
web.shutdown()

So after running this example you'll have two Video URL sources, one is default http://localhost:8000/video and other one is http://localhost:8000/video2, so you can change/modify your index.html HTML file to point to these two Video URL sources:

https://github.com/abhiTronix/vidgear-vitals/blob/2ff1ee33b55d0a71c8e7188161ace94751db6112/webgear/templates/index.html#L24-L26

Goodluck!!!

abhiTronix commented 3 years ago

@PrashantSaikia Please test this example as sson as possible and let us know how it went.

abhiTronix commented 3 years ago

Marked solved, as working as intended. Feel free to test it and provide valuable feedback when possible.

PrashantSaikia commented 3 years ago

Hi there. I'm a little unclear on where do I need to put this index.html, or how to run it. Do I need to run the code you wrote above, and then separately run this index.html, or something like that? If so, its getting a bit complicated. Also, I actually have done this with websockets; however, after doing pose estimation on the video feeds, the output rendered is very choppy (even with the answer marked as the solution). So, the whole reason why I was trying out Webgear was in the hope that I might get better FPS with Webgear. Do you think that's going to be the case?

abhiTronix commented 3 years ago

Hi there. I'm a little unclear on where do I need to put this index.html, or how to run it. Do I need to run the code you wrote above, and then separately run this index.html

@PrashantSaikia You need to look at docs on how WebGear works. WebGear dynamically checks for HTML, CSS, JS files at runtime with its Auto-Generation workflow. So you need to change HTML file generated at Default Location to your use case.

abhiTronix commented 3 years ago

So, the whole reason why I was trying out Webgear was in the hope that I might get better FPS with Webgear. Do you think that's going to be the case?

@PrashantSaikia It depends upon how fast source is outputting the frames. If source itself is slow, then WebGear can't do much. See WebGear's Performance Enhancements. Also, you can try WebGear_RTC that utilizes much faster WebRTC technology in the similar way.

PrashantSaikia commented 3 years ago

Ok, so I saved the index.html as follows:

{% extends "base.html" %}
{% block content %}
    <h1 class="glow">WebGear Video Feed</h1>
    <div class="rows">
      <video id="myvideo" width="320" height="240" controls style="background:black">
        <source class="active" src="http://localhost:8000/video" type="video/mp4" />
        <source src="http://localhost:8000/video2" type="video/mp4" />
      </video>
    </div>
{% endblock %}

and used options to point to the location where it is saved (which I've kept as ./ for convenience as of now). After now running your code, I can see the two html files in the folder, but it doesn't display the two videos side by side - it still shows only one, which I guess probably means they are overlapping, which I pointed out in my original post.

abhiTronix commented 3 years ago

I can see the two html files in the folder, but it doesn't display the two videos side by side

@PrashantSaikia What two html files?

PrashantSaikia commented 3 years ago

index.html and base.html

abhiTronix commented 3 years ago

(which I've kept as ./ for convenience as of now)

@PrashantSaikia Kindly enable debugging(logging=True) and Paste the terminal log here. In my opinion ./ is not working but still using old files located at .vidgear at your home directory.

abhiTronix commented 3 years ago

index.html and base.html

@PrashantSaikia That is wrong, it should not be two files but it must also contain 404.html and 500.html, otherwise WebGear triggers Auto-Generation process that will overwrite old files.

PrashantSaikia commented 3 years ago

Oh ok, I copied the 404.html and 500.html files too into the folder. Still no difference in result. Could you please correct my index.html and tell me exactly what to do after that? I just want to see if there's any improvement in FPS or not.

abhiTronix commented 3 years ago

Oh ok, I copied the 404.html and 500.html files too into the folder. Still no difference in result.

@PrashantSaikia Are you able to achieve two streams on your browser? You're not helping me with debug output, I can't help with your problem.

abhiTronix commented 3 years ago

@PrashantSaikia Just paste whatever is in your terminal after running the python script.

PrashantSaikia commented 3 years ago

No, I'm not able to achieve two streams in my browser. Here's what is in my terminal after running the code:

10:28:54 :: Helper Asyncio :: DEBUG :: Directory already exists at `/Users/username/Documents/OpenCV_Project/.vidgear/webgear`
10:28:54 :: Helper Asyncio :: DEBUG :: Directory already exists at `/Users/username/Documents/OpenCV_Project/.vidgear/webgear/static`
10:28:54 :: Helper Asyncio :: DEBUG :: Directory already exists at `/Users/username/Documents/OpenCV_Project/.vidgear/webgear/templates`
10:28:54 :: Helper Asyncio :: DEBUG :: Directory already exists at `/Users/username/Documents/OpenCV_Project/.vidgear/webgear/static/js`
10:28:54 :: Helper Asyncio :: DEBUG :: Directory already exists at `/Users/username/Documents/OpenCV_Project/.vidgear/webgear/static/css`
10:28:54 :: Helper Asyncio :: DEBUG :: Directory already exists at `/Users/username/Documents/OpenCV_Project/.vidgear/webgear/static/img`
10:28:54 :: Helper Asyncio :: DEBUG :: Found valid WebGear data-files successfully.
10:28:54 :: WebGear :: DEBUG :: `/Users/username/Documents/OpenCV_Project/.vidgear/webgear` is the default location for saving WebGear data-files.
10:28:54 :: WebGear :: DEBUG :: Setting params:: Size Reduction:20%, JPEG quality:90%, JPEG optimizations:False, JPEG progressive:False.
10:28:54 :: WebGear :: WARNING :: Given source is of NoneType!
10:28:54 :: WebGear :: DEBUG :: Initiating Video Streaming.
10:28:54 :: WebGear :: DEBUG :: Running Starlette application.
INFO:     Started server process [67988]
INFO:     Waiting for application startup.
INFO:     Application startup complete.
INFO:     Uvicorn running on http://localhost:8000 (Press CTRL+C to quit)
INFO:     ::1:51917 - "GET / HTTP/1.1" 200 OK
INFO:     ::1:51918 - "GET /video HTTP/1.1" 200 OK
INFO:     ::1:51917 - "GET /static/css/custom.css HTTP/1.1" 304 Not Modified
INFO:     ::1:51919 - "GET /static/js/custom.js HTTP/1.1" 304 Not Modified
INFO:     ::1:51917 - "GET /video HTTP/1.1" 200 OK
abhiTronix commented 3 years ago

10:28:54 :: WebGear :: DEBUG :: /Users/username/Documents/OpenCV_Project/.vidgear/webgear is the default location for saving WebGear data-files.

@PrashantSaikia Ok, the default location have changed successfully.

INFO: Uvicorn running on http://localhost:8000 (Press CTRL+C to quit) INFO: ::1:51917 - "GET / HTTP/1.1" 200 OK INFO: ::1:51918 - "GET /video HTTP/1.1" 200 OK INFO: ::1:51917 - "GET /static/css/custom.css HTTP/1.1" 304 Not Modified INFO: ::1:51919 - "GET /static/js/custom.js HTTP/1.1" 304 Not Modified INFO: ::1:51917 - "GET /video HTTP/1.1" 200 OK

But Still no /video2 here. Did you changed index html? Also what happens when you visit http://localhost:8000/video and http://localhost:8000/video2 (open both in different tabs), did you see output simuntaneously on both?

PrashantSaikia commented 3 years ago

Did you changed index html?

Yes, I copy-pasted my index.html above. Pasting it again here:

{% extends "base.html" %}
{% block content %}
    <h1 class="glow">WebGear Video Feed</h1>
    <div class="rows">
      <!-- <div id="imgcontainer" onclick="toggleFullScreen('imgcontainer')"><img src="/video" alt="Feed" title="click to go fullscreen"></div> -->
      <video id="myvideo" width="320" height="240" controls style="background:black">
        <source class="active" src="http://localhost:8000/video" type="video/mp4" />
        <source class="active" src="http://localhost:8000/video2" type="video/mp4" />
      </video>
    </div>
{% endblock %}

Also what happens when you visit http://localhost:8000/video and http://localhost:8000/video2, did you see output?

Yes, I checked. When I give the video path as webcam for both the streams, it doesn't display anything on the browser. So I changed the video paths to two pre-downloaded mp4 videos. After that, when I run the script and visit http://localhost:8000/video and http://localhost:8000/video2, it downloads the two video files, but without any extension. So they get downloaded as Document files, and even though they are the same size as my mp4 files (15-20 MB), they appear as a static image upon opening them and can't be played as video.

abhiTronix commented 3 years ago
{% extends "base.html" %}
{% block content %}
  <h1 class="glow">WebGear Video Feed</h1>
    <div class="rows">
      <!-- <div id="imgcontainer" onclick="toggleFullScreen('imgcontainer')"><img src="/video" alt="Feed" title="click to go fullscreen"></div> -->
      <video id="myvideo" width="320" height="240" controls style="background:black">
        <source class="active" src="http://localhost:8000/video" type="video/mp4" />
        <source class="active" src="http://localhost:8000/video2" type="video/mp4" />
      </video>
    </div>
{% endblock %}

@PrashantSaikia Also your html code is wrong. Source is of img type not video. So change it to following:

{% block content %}
    <h1 class="glow">WebGear Video Feed</h1>
    <div class="rows">
       <img src="/video" alt="Feed1">
       <img src="/video2" alt="Feed2">
    </div>
{% endblock %}
PrashantSaikia commented 3 years ago

Ok, done. Still see only one stream in the browser:

Screenshot 2021-06-24 at 10 52 33 AM
abhiTronix commented 3 years ago

Yes, I checked. When I give the video path as webcam for both the streams, it doesn't display anything on the browser. So I changed the video paths to two pre-downloaded mp4 videos. After that, when I run the script and visit http://localhost:8000/video and http://localhost:8000/video2, it downloads the two video files, but without any extension. So they get downloaded as Document files, and even though they are the same size as my mp4 files (15-20 MB), they appear as a static image upon opening them and can't be played as video.

@PrashantSaikia That's normal behavior since WebGear uses motion-jpeg, that display images frame by frame and not a valid video container.

abhiTronix commented 3 years ago

Ok, done. Still see only one stream in the browser:

Screenshot 2021-06-24 at 10 52 33 AM

@PrashantSaikia What source are you using? I'm getting correct out put here: image

PrashantSaikia commented 3 years ago

@PrashantSaikia What source are you using? I'm getting correct out put here:

I am using pre-downloaded videos, the same ones from here.

Also, like I mentioned before, if I give the video path as 0 (i.e., webcam), it doesn't display anything in the browser.

abhiTronix commented 3 years ago

@PrashantSaikia Ok I'm uploading complete Zip file here with everything working, Please wait.

abhiTronix commented 3 years ago

Also, like I mentioned before, if I give the video path as 0 (i.e., webcam),

@PrashantSaikia Are you sure? OpenCV works with camera out of the box. Try -1 integer value at input.

PrashantSaikia commented 3 years ago

@PrashantSaikia Are you sure? OpenCV works with camera out of the box. Try -1 integer value at input.

Ok, tried this as well just now. Now there's a "?" that appears in the div where the video is supposed to display. And the terminal output also says that the path is wrong:

OpenCV: out device of bound (0-0): -1
OpenCV: camera failed to properly initialize!

So I guess 0 is the correct path for webcam. When I give 0, there's nothing that appears in the video placeholder/div - no question mark, no video.

abhiTronix commented 3 years ago

@PrashantSaikia See its working: image

And here is the working zip file:

webgear_example.zip

Just do python test_example.py inside extracted folder to run it.

abhiTronix commented 3 years ago

So I guess 0 is the correct path for webcam. When I give 0, there's nothing that appears in the video placeholder/div - no question mark, no video.

@PrashantSaikia Your code must be wrong, just use mine with 0 value. Goodluck!

PrashantSaikia commented 3 years ago

And here is the working zip file:

Ok, now it shows the two videos one after the other. Upon refreshing the page though, the two videos play side by side as desired. And also, the videos are playing at a faster speed, looks like 2-3x from eyeballing.

Here, I did a screen recording:

https://user-images.githubusercontent.com/39755678/123198034-07e25100-d4df-11eb-9eac-63d5085c6ef8.mp4

And giving the webcam still produces blank output:

Screenshot 2021-06-24 at 11 27 22 AM

I guess this might be a Windows vs MacOS issue then, since you're getting the desired output but I'm not.

abhiTronix commented 3 years ago

I guess this might be a Windows vs MacOS issue then, since you're getting the desired output but I'm not.

I'm also on windows. Try this example:

# import required libraries
from vidgear.gears import CamGear
import cv2

# Open suitable video stream, such as webcam on first index(i.e. 0)
stream = CamGear(source=0).start() 

# loop over
while True:

    # read frames from stream
    frame = stream.read()

    # check for frame if Nonetype
    if frame is None:
        break

    # {do something with the frame here}

    # Show output window
    cv2.imshow("Output", frame)

    # check for 'q' key if pressed
    key = cv2.waitKey(1) & 0xFF
    if key == ord("q"):
        break

# close output window
cv2.destroyAllWindows()

# safely close video stream
stream.stop()

What do you see?

abhiTronix commented 3 years ago

And also, the videos are playing at a faster speed, looks like 2-3x from eyeballing.

@PrashantSaikia You can play with WebGear's Performance Enhancements for further performance boost. Also, you can try WebGear_RTC that utilizes much faster WebRTC technology in the similar way.

PrashantSaikia commented 3 years ago

I'm also on windows.

Yeah, that's why I said, because I'm on Mac.

What do you see?

I can see the webcam feed. When displaying a single video, its not a problem. In fact, I had already ported my pose recognition module into your example code before I made this post, and it works great. Its just that making two videos play side by side is proving difficult.

abhiTronix commented 3 years ago

Its just that making two videos play side by side is proving difficult.

@PrashantSaikia This is making no sense. If you're having no problem with single video, then you must have no problem with any number of video. Can you update your vidgear to latest as follows:

  # clone the repository and get inside
  git clone https://github.com/abhiTronix/vidgear.git && cd vidgear

  # checkout the latest testing branch
  git checkout testing

  # install with asyncio support
  pip install .[asyncio]

and then try.

abhiTronix commented 3 years ago

@PrashantSaikia Also, are you using 0 index this time? Double check it is 0 and not -1. Also problem with OpenCV is that it sometimes doesn't display the actual problem.

abhiTronix commented 3 years ago

@PrashantSaikia Even my camera is working without any problem:

image

Here's code:

webgear_example.zip

PrashantSaikia commented 3 years ago

@PrashantSaikia Also, are you using 0 index this time? Double check it is 0 and not -1.

Yes, I'm using 0. I never use -1, except now to test it.

Also problem with OpenCV is that it sometimes doesn't display the actual problem.

Yes, that's true unfortunately.

Who knew rendering the two videos on the browser instead of on two popup windows as OpenCV does would be such a time consuming and energy-intensive task. I just wish OpenCV had an inbuilt function to render the results on browser, sigh.

ahmedosman2001 commented 3 years ago

Hi, i tested the zip file you shared and it also works with two webcams. Now, instead of using WebGear, i tried to use WebGear_RTC but I'm keep getting this error when i visit http://localhost:8000/video2. TypeError: 'coroutine' object is not an iterator

This is the debug and full error

Screenshot 2021-07-11 at 16 36 06

This is my code (Since WebGear and WebGear_RTC very similar, i didn't change a lot of stuff from your zip file )

#import libs
import uvicorn, asyncio, cv2
from starlette.routing import Route
from vidgear.gears.asyncio import WebGear_RTC
from vidgear.gears.asyncio.helper import reducer
from starlette.responses import StreamingResponse
from aiortc import VideoStreamTrack
from av import VideoFrame

options = {"custom_data_location": "./"}
#initialize WebGear app without any source
web = WebGear_RTC(logging=True,  **options)

class Custom_RTCServer(VideoStreamTrack):
    """
    Custom Media Server using OpenCV, an inherit-class
    to aiortc's VideoStreamTrack.
    """

    def __init__(self, source=None):

        # don't forget this line!
        super().__init__()

        # initialize global params
        self.stream = cv2.VideoCapture(0)
        self.stream2 = cv2.VideoCapture(1)

    async def recv(self):
        """
        A coroutine function that yields `av.frame.Frame`.
        """
        # don't forget this function!!!

        # get next timestamp
        pts, time_base = await self.next_timestamp()

        # read video frame
        (grabbed, frame) = self.stream.read()

        # if NoneType
        if not grabbed:
            return None

        # reducer frames size if you want more performance otherwise comment this line
        frame = await reducer(frame, percentage=30)  # reduce frame by 30%

        # contruct `av.frame.Frame` from `numpy.nd.array`
        av_frame = VideoFrame.from_ndarray(frame, format="bgr24")
        av_frame.pts = pts
        av_frame.time_base = time_base

        # return `av.frame.Frame`
        return av_frame

    async def recv2(self):
        """
        A coroutine function that yields `av.frame.Frame`.
        """
        # don't forget this function!!!

        # get next timestamp
        pts, time_base = await self.next_timestamp()

        # read video frame
        (grabbed, frame) = self.stream2.read()

        # if NoneType
        if not grabbed:
            return None

        # reducer frames size if you want more performance otherwise comment this line
        frame = await reducer(frame, percentage=30)  # reduce frame by 30%

        # contruct `av.frame.Frame` from `numpy.nd.array`
        av_frame = VideoFrame.from_ndarray(frame, format="bgr24")
        av_frame.pts = pts
        av_frame.time_base = time_base

        #return `av.frame.Frame`
        return av_frame

    async def custom_video_response(self, scope):
        """
        Return a async video streaming response for `my_frame_producer2` generator
        """
        assert scope["type"] in ["http", "https"]
        await asyncio.sleep(0.00001)
        return StreamingResponse(
            self.recv2(),
            media_type="multipart/x-mixed-replace; boundary=frame",
        )

    def terminate(self):
        """
        Gracefully terminates VideoGear stream
        """
        #don't forget this function!!!

        #terminate
        if not (self.stream is None):
            self.stream.release()
            self.stream = None
        if not (self.stream2 is None):
            self.stream2.release()
            self.stream2 = None

Object = Custom_RTCServer()
web.config["server"] = Custom_RTCServer()

#append new route i.e. new custom route with custom response
web.routes.append(
     Route("/video2", endpoint=Object.custom_video_response)
     )

#run this app on Uvicorn server at address http://localhost:8000/
uvicorn.run(web(), host="localhost", port=8000)

#close app safely
web.shutdown()

What I'm trying to do is use WebGear_RTC to stream two cameras at the same time. Thanks

abhiTronix commented 3 years ago

@ahmedosman2001 It will not work by simply changing python code alone in WebGear_RTC since its backend aiortc library is pretty complex and need both frontend javascript and backend python code to be altered simuntaneously and synchronized properly. Since javascripting is not forte, so I can't really help you, but you can look into this issue: https://github.com/aiortc/aiortc/issues/382

However, you can do one more thing, that you can concatenate two frames horizontally using numpy and then send it to peer, as follows:

# import necessary libs
import uvicorn, asyncio, cv2
import numpy as np
from av import VideoFrame
from aiortc import VideoStreamTrack
from vidgear.gears.asyncio import WebGear_RTC
from vidgear.gears.asyncio.helper import reducer

# initialize WebGear_RTC app without any source
web = WebGear_RTC(logging=True)

def get_conc_frame(frame1, frame2):
    h1, w1 = frame1.shape[:2]
    h2, w2 = frame2.shape[:2]

    # create empty matrix
    vis = np.zeros((max(h1, h2), w1 + w2, 3), np.uint8)

    # combine 2 frames
    vis[:h1, :w1, :3] = frame1
    vis[:h2, w1 : w1 + w2, :3] = frame2

    return vis

# create your own Bare-Minimum Custom Media Server
class Custom_RTCServer(VideoStreamTrack):
    """
    Custom Media Server using OpenCV, an inherit-class
    to aiortc's VideoStreamTrack.
    """

    def __init__(self, source1=None, source2=None):

        # don't forget this line!
        super().__init__()

        # check is source are provided
        if source1 is None or source2 is None:
            raise ValueError("Provide both source")

        # initialize global params
        # define both source here
        self.stream1 = cv2.VideoCapture(source1)
        self.stream2 = cv2.VideoCapture(source2)

    async def recv(self):
        """
        A coroutine function that yields `av.frame.Frame`.
        """
        # don't forget this function!!!

        # get next timestamp
        pts, time_base = await self.next_timestamp()

        # read video frame
        (grabbed1, frame1) = self.stream1.read()
        (grabbed2, frame2) = self.stream2.read()

        # if NoneType
        if not grabbed1 or not grabbed2:
            return None

        # concatenate frame
        frame = get_conc_frame(frame1, frame2)

        # reducer frames size if you want more performance otherwise comment this line
        frame = await reducer(frame, percentage=30)  # reduce frame by 30%

        # contruct `av.frame.Frame` from `numpy.nd.array`
        av_frame = VideoFrame.from_ndarray(frame, format="bgr24")
        av_frame.pts = pts
        av_frame.time_base = time_base

        # return `av.frame.Frame`
        return av_frame

    def terminate(self):
        """
        Gracefully terminates VideoGear stream
        """
        # don't forget this function!!!

        # terminate
        if not (self.stream1 is None):
            self.stream1.release()
            self.stream1 = None

        if not (self.stream2 is None):
            self.stream2.release()
            self.stream2 = None

# assign your custom media server to config with both adequate sources (for e.g. foo1.mp4 and foo2.mp4)
web.config["server"] = Custom_RTCServer(source1="foo.mp4", source2="foo.mp4")

# run this app on Uvicorn server at address http://localhost:8000/
uvicorn.run(web(), host="localhost", port=8000)

# close app safely
web.shutdown()
abhiTronix commented 3 years ago

@ahmedosman2001

And here is the working zip file with frame concatenation trick:

webgear_rtc_example.zip

Just do python test_example.py inside extracted folder to run it.

ahmedosman2001 commented 3 years ago

I tested earlier and it works fine when streaming video file. However, i couldn't make it work for two webcams. For example, when the sources is changed from foo.mp4 to webcam (0 and 1), it just displays black screen. Thanks.

abhiTronix commented 3 years ago

@ahmedosman2001 It's working fine on my end. And If it works with video files, webcam is no exception. Sort this out yourself.

abhiTronix commented 3 years ago

@PrashantSaikia @ahmedosman2001 This might be error why two cameras are not working with OpenCV: https://stackoverflow.com/questions/29664399/capturing-video-from-two-cameras-in-opencv-at-once

This solution would be to use multithreaded CamGear/VideoGear instead of OpenCV in your code. goodluck!