GodotVR / godot-xr-tools

Support scenes for AR and VR in Godot
MIT License
470 stars 64 forks source link

Add mipmaps and more filtering options for Viewport2Din3D #459

Open da-molchanov opened 1 year ago

da-molchanov commented 1 year ago

Current behavior: Viewport2Din3D takes the Viewport texture and draws it on a quad with either nearest or linear filtering. That texture doesn't have mipmaps. The quad looks fine up close, but as you move away, it starts aliasing heavily. This is especially noticeable in VR due to slight camera movements, and is distracting.

Desired behavior: Have an option to generate the mipmaps when the Viewport texture is updated, and display it with an appropriate filter to reduce aliasing. Have an option to use an RGSS shader to reduce the blur introduced by the filtering, and improve the clarity further.

This is discussed in detail in this blog post by Ben Golus. I have played with this technique in Unreal Engine, and it looks great without being too expensive.

I tried to make a proof of concept implementation. When enabled for all Viewport2Din3D's in the Godot XR Tools Demo project, it runs smoothly on Quest 2 with the gl_compatibility renderer even when I regenerate the mipmaps every frame. I can do a proper implementation and some benchmarks next week.

Some discussion points:

  1. This is pretty cheap if the texture is only updated occasionally and the resolution is not too high. It can get expensive if the textures are large, continually animated, or if there are just too many of them. Image::generate_mipmaps runs on the CPU, and it could be worth offloading to a separate thread. Alternatively, it could be done on the GPU (e.g., Unreal Engine's Android Vulkan renderer does runtime mipmap generation in a compute shader); looks like it can also be done with Vulkan blits or OpenGL's glGenerateMipmap, but I don't have experience with those. Does Godot have a built-in way to generate mipmaps on the GPU? Am I missing a better way to get mipmaps for the Viewport texture?
  2. Another way to improve UI clarity is to use compositor layers, but those can be a pain to use, especially when something needs to be drawn on top of them. Are compositor layers supported in Godot?
  3. In addition to the already present nearest filtering, smooth nearest filtering could also be useful here.

Current:

https://user-images.githubusercontent.com/20925473/234992053-5428508f-f112-47e6-a80e-b8442e1e4330.mp4

Proposed:

https://user-images.githubusercontent.com/20925473/234991999-ff9ab4eb-9179-4231-a786-e006d3a1e3ae.mp4

Malcolmnixon commented 10 months ago

Pull Request #493 allows the user to specify their own material for the viewport, and that material can specify advanced sampling options such as "Linear Mipmap Anisotropic".

BastiaanOlij commented 10 months ago

@Malcolmnixon thats sadly not enough, you actually have to create the mipmaps as suggested in the OP. I just don't know if this can be combined with viewports as the render target isn't created with mipmap support. That may require an upstream change in Godot itself.

BastiaanOlij commented 10 months ago

Raised a proposal for Godot to add mipmap support on viewports:

https://github.com/godotengine/godot-proposals/issues/7576

BastiaanOlij commented 10 months ago

@da-molchanov the videos didn't work for me btw, any chance you can upload them on Youtube or something?