Website Stuck When Displaying Multiple Camera Streams Simultaneously

[dan246]

          @dan246 

Thank you for your very useful sharing I have solved the problem of negative time values and stuck recognation module. The cause is the time.time() function. When we use time.time() which can jump if the system clock is adjusted. Use time.monotonic() instead of time.time() for more accurate duration measurements. You should switch to using time.monotonic() New Problem: I want to view multiple cameras at the same time on one Web Browser Tab, so I create an html page with 8 Image tags then display 8 camera stream is well. However, when opening this tab sometime, the website stuck and not displaying the camera. Even when opening it in another tab. Problem is fix when i refresh the old tab and the system run normally. I think Flask server stuck when there are too many requests per session, do you have any suggestions on how to fix it?

Originally posted by @xxtkidxx in https://github.com/dan246/VisionFlow/issues/4#issuecomment-2497644039

[dan246]

Hi @xxtkidxx I’ve moved your new question here to better track and address the issue. Thank you for sharing the details!

Thank you for the suggestion! I’ve already switched to using time.monotonic() in the dev branch. Regarding the new issue, I recommend starting with Gunicorn for better performance. Here are the details: Install Gunicorn You can install Gunicorn using pip:

pip install gunicorn

Run Flask with Gunicorn

Replace the Flask development server with Gunicorn for production use. Use the following command:

gunicorn -w 4 -k gevent -b 0.0.0.0:5000 --timeout 120 app:app

Options explained:

• -w 4: Specifies the number of worker processes. Adjust this based on your server's CPU cores.
• -k gevent: Enables asynchronous workers, allowing efficient handling of multiple requests simultaneously.
• --timeout 120: Prevents Gunicorn from prematurely killing long-running requests.

Optimize Your Code (Still to be Tested)

Replace time.sleep with await asyncio.sleep to ensure non-blocking execution. This is especially important for routes like /recognized_stream/<camera_id>.

These steps should improve your application's performance and help prevent the server from getting stuck when handling a high volume of requests. Let me know if you need further assistance!

[xxtkidxx]

Hi @dan246 I installed it as you suggested. I see performance improvement but I still can't open 10 cameras on the same tab.

[dan246]

Hi @xxtkidxx Thank you for sharing the update! To better understand and help resolve the issue, could you provide the following details about your setup?

1. Gunicorn Configuration The number of workers (-w flag) and worker connections (--worker-connections flag), if specified.

2. Resource Usage (Using htop) While the issue is occurring, you can run the htop command in your terminal to monitor CPU and memory usage in real time. Specifically, look for:

Worker Process Usage: Are some workers maxing out CPU or memory? System Load: Is the load average too high relative to the number of CPU cores? Overall Memory Usage: Is the system running out of available memory?

[xxtkidxx]

@dan246 Thanks for quick response My system is as follows: Number of Camera: 10 FPS: 5 Number of Worker Container: 5 Website Stuck When Displaying 6th Camera Streams Simultaneously. I have tried switching to other APIs like FASTAPI, AIOHTTP but the results are still the same. Note that I open multiple cameras on the same Chrome tab. Docker can not log 7th 8th Camera recognized Request

[dan246]

Hi @xxtkidxx Sorry for the late reply. After reviewing the configuration you provided, I think you can try the following steps:

1. Increase the number of Gunicorn workers: Increase the number of Gunicorn workers to allow the server to handle more requests. For example: gunicorn -w 10 -k gevent -b 0.0.0.0:5000 --timeout 120 app:app

• -w 10: Increase the number of workers from 4 to 10 (adjust based on the number of CPU cores).

From the image you shared:

The CPU usage has already reached 71%. The GPU utilization is also at 78%.

2. You might also consider lowering the FPS (to 2~3) to reduce the load.

PS. If the above adjustments don't work, check the browser's connection limit:

• Open your browser's developer tools (F12) -> Network tab.

• Reload the page and observe the requests for the camera streams:

• Check if /recognized_stream/<ID> shows some requests in a Pending state, which indicates the browser didn't send the requests due to connection limits.

• If you see some requests marked as Blocked or similar, it might be related to the browser's connection limit.

Note: Since my project doesn't include Nginx configuration, I suspect the issue might be with the browser's connection limits.

Please try these steps and let me know the results, thanks !

[xxtkidxx]

Hi @dan246 I have tried many ways and setup on different computers but cannot fix this problem. I think it's a limitation of Flask, it doesn't seem to be designed to stream videos with so many request on one tab I built another solution that combines camera photos. Everything seems fine I wish there was a better solution for parallel processing of videos using ultralytics. Do you have a better solution than using batch images and a for loop?

[dan246]

Hi @xxtkidxx Sorry for the late reply. At the moment, I don’t have a better solution than the current approach, but I’m actively exploring alternatives and will keep updated on any new findings.

If you have any suggestions or ideas, I’d be happy to discuss them as well!

Thank you for your understanding.