roninpawn
commented
2 months ago I think you'll need to say that again, because it doesn't quite make sense to me.
OpenCV's video access for Python is a single-threaded process that employs a method of acquiring frames which does not necessarily return the correct frame, when asked for either by the frame number or by Hours:Minutes:Seconds.milliseconds. If you have to move the read-head at all (and presuming it matters whether you get the actual frame/time back that you asked for) then OpenCV is not a viable solution.
In addition to this, since OpenCV's accessing method is single threaded, and CANNOT BE multithreaded due to its inability to move the read head correctly - it is something like 20x slower than the multithreaded access that FFmpeg provides. (which is what this script enables: frame acquisition by FFmpeg)
Now, if what you're saying is - on Linux - OpenCV just opens a pipe to FFmpeg to acquire the frames, then its doing exactly the same thing my script does. BUT! If that's the case, then there really shouldn't be any room for a "3x faster" improvement in performance... A pipe is a pipe is a pipe. Whether OpenCV asks for it, or my script asks for it, the same data comes through, at the same rate. So the only way that one would be slower than the other, would have to be due to how you implemented my VideoStream() class.
To field a guess: I would figure that when you implemented MY class, you were asking for the frames in a different format then what you needed, and then you shoved that frame through a costly transformation matrix. Whereas, when you implemented OpenCV, it probably asked FFmpeg to give it the frames in the format it wanted, without requiring any configuration from you.
But that's just a guess. And I'm not 100% that I'm sure what you're saying.
For further reference... Here's the reason that my VideoStream class, and so many other FFmpeg-wrappers for Python, exist: https://github.com/opencv/opencv/issues/9053 ...because OpenCV doesn't handle video correctly. In that thread, you can find the maintainers of OpenCV themselves recommending that developers find a better solution (like FFmpeg) if you're writing an app that needs more than just a shallow example showing that frames of video can be processed using OpenCV.
It's slower than using cv2.Videocapture if you're looking to actually get a compatible np array to do anything with. Maybe if your application is different but using my self compile ffmpeg (which cv2 uses by default on linux, on windows it'll download a binary and use a pre-compile one which may not be using Cuda or otherwise) and cv2 is 3x faster than this to get me from the read to the correct cv2 image. If you're application is like mine then just use cv2.VideoCapture and make sure cv2 is using the right ffmpeg. It's about as fast as you're gonna get.