search

jaseg / python-mpv

Python interface to the awesome mpv media player
https://git.jaseg.de/python-mpv.git
Other
543 stars 68 forks source link

Scale overlay image size without increasing resolution, to get large pictures and high performance #135

Open jkazan opened 3 years ago

jkazan commented 3 years ago

I am updating an overlay every frame, and to_bytes inside overlay.update is the bottleneck here. I do not mind if the DPI of the overlay image is low, but I need large images to be updated fast.

A fix for this would be fantastic!

Thanks for all the good work!

jaseg commented 3 years ago

I am fairly sure that this is not currently possible, since libmpv does not seem to expose any API to do that. You could try to optimize ImageOverlay's update() implementation, but I haven't looked into that more closely yet.

I think a better way to get the same result might be to use mpv's OpenGL API from python, and manually render the overlay on top of mpv's framebuffer using OpenGL.

jkazan commented 3 years ago

I found the --drm-draw-surface-size paramter at https://mpv.io/manual/master/#video-output-drivers Isn't this what I am looking for ? However, whatever I set the resolution to, nothing changes. Perhaps I am misinterpreting? My understanding is that the overlay resolution will be <WxH> and then scaled up to the resolution of the mpv window.

Could you please clarify this? I really appreciate your help!

jaseg commented 3 years ago

That option is specific to the DRM (Linux Direct Rendering Manager) video output driver. Unless you are rendering on a console linux setup that does not have X11 or wayland you will use a different video output driver such as gpu or direct3d.

jkazan commented 3 years ago

I see. Any chance of doing the same thing with X11?

jaseg commented 3 years ago

Well, using OpenGL would basically amount to this. Via the gpu video output driver, OpenGL is also the default video output anyway, so integrating against libmpv's OpenGL API should not pose any significant compatibility issues either (at least for desktops/laptops, rpi&co might not perform as well).

jkazan commented 3 years ago

Cool, I'll look into it. Thanks a lot for your quick replies!

jkazan commented 3 years ago

It seems I can use the property vf="gpu=400:225", but that only affects the video, not the overlay. Any ideas?

jaseg commented 3 years ago

The only suggestion I can come up with is to use opengl to properly control this. For an opengl/python-mpv example, please have a look at https://gist.github.com/jaseg/657e8ecca3267c0d82ec85d40f423caa and https://gist.github.com/cosven/b313de2acce1b7e15afda263779c0afc

vf=gpu inserts an OpenGL/GPU video filter stage. This is run before the video is passed on to the video output driver, which is pasting in any overlays. In your case, that filter stage does nothing and just passes on the framebuffer that it gets at the resolution you specify. The --glsl-shaders option offers a weird command line/option interface with that but I would really advise against that.