sandersdan
commented
2 years ago Readback of hardware VideoFrames is inherently expensive. At 1080p I am not particularly surprised that decoding via software into CPU-attached memory is more efficientoverall While I do expect improvements to Chrome's readback implementation over time (mostly by making it more parallel), I'm not sure that it will ever be faster than software decode for this use case.
The preferred use of hardware frames is to use shaders to process them. texImage2D() is likely much faster than readback, and even better will be importExternalTexture() in WebGPU.
When I use NVIDIA geforce gt710, It decodes 1080p video and renders it through canvas, which is about 80fps, while g750m is 400fps. We think this is an obvious difference caused by the performance of GPU.
However, when we try to copy the video frame data out for post-process(such as video effects, transition etc.), it drops from 400fps to 80fps, and the performance drops too seriously. Is there a way to copy it quickly or provide a C/C++ interface, so we can quickly obtain video frame data by WebAssembly?
In addition, if the speed of CPU decoding is faster than that of webcodecs decoding and delivering to the native layer, WebCodecs can only be used for video playback, and cann't meet the performance requirements of video editing.It's really a confusing problem！