glenn-jocher
commented
3 weeks ago Hi there! š Thanks for bringing up your concerns regarding OpenCV's performance with video processing and the fps drop issue related to cv2.waitKey(). OpenCV is indeed a foundational element in many computer vision projects, but its limitations, especially on certain platforms and configurations, are recognized.
Switching from OpenCV to other libraries like PyQt6 or utilizing GPU-accelerated rendering through packages that exploit Vulkan or DirectX for rendering is an interesting proposition. We acknowledge the potential benefits in performance and device support improvements it could offer, especially for video streaming and real-time applications.
Regarding the libraries you mentioned, PyQt6 offers a robust framework for GUI applications but might introduce a heavier dependency compared to OpenCV for straightforward video capture and rendering tasks. On the other hand, vidgear provides a fascinating approach, especially with its support for multi-threaded video streams which can help mitigate the fps drop issue.
For now, Ultralytics is closely integrated with OpenCV, but we're always exploring ways to enhance YOLOv8's performance and usability across different platforms. Your suggestions are valuable and might be considered in future updates or alternative branches for specific use cases like high-performance streaming or real-time applications.
We encourage the community to experiment with alternative methods and share their findings or even contribute back. It's through insights like yours that we can continue to evolve and improve. Thank you for your input! š
Search before asking
Description
OpenCV is not prepared for videos: if cv2.waitKey(1) & 0xFF == ord("q") drops 50% of fps and more in windows. https://forum.opencv.org/t/fps-drop-due-to-the-waitkey-function/3717 https://stackoverflow.com/questions/60543088/capturing-fps-drops-with-lower-resolution-opencv https://forum.opencv.org/t/confusing-waitkey-buffering-behavior/10058 https://answers.opencv.org/question/52774/waitkey1-timing-issues-causing-frame-rate-slow-down-fix/ https://dsp.stackexchange.com/questions/44129/does-the-cv2-waitkey-argument-determine-the-frame-rate-while-capturing-a-video
Also, waitkey It dependes of scheduling internally of the SO. https://pypi.org/project/PyQt6/ or https://github.com/fastplotlib/fastplotlib it could be a great solution. Why? It uses pygfx( vulkan or metal or directx12) for rendering. Because use vulkan-python is more complicated.
Example: if you mantain press a key of keyboards, inference will be faster Also for video and cameras streaming you can read in a new frame and give more support devices with vidgear. https://github.com/abhiTronix/vidgear
Use case
Demos... streamings etc
Additional
No response
Are you willing to submit a PR?