Moving Post-Processing to Compute

[Nolram12345]

Is your feature request related to a problem? Please describe. When testing & deploying Filament-based rendering across devices, performance quickly becomes a limiting factor when rendering model or scenes with higher complexity, or rendering at higher resolutions. This is especially apparent on modern phones & desktops, where resolutions in the 1440p-4k range are very common. After extensive profiling, we've found that a large bottleneck currently is post-processing, especially SSAO. Due to being run synchronously and in fragment, there are many scenarios where SSAO alone can take up almost half the entire frametime, especially on mobile devices where raster throughput is even more limited than on Desktop with dedicated GPUs.

Note: Screen-Space Reflections in particular seem to have some other performance issues unrelated to raster throughput.

Describe the solution you'd like A re-implementation/port of Filament's post processing, especially SSAO, to Compute shaders, allowing for higher efficiency and faster rendering on almost every device. This would also allow for running SSAO in async compute, cutting frametimes nearly in half in some tests. Compute Shaders are supported on almost all semi-modern devices, and on both OpenGL (GLES 3 and above) and Vulkan.

Describe alternatives you've considered While it's possible to skip post processing or run it at a lower resolution, this results in dramatic visual downgrades across the board. Compute optimisation could help all post-processing effects employed by Filament, including very fundamental ones like tonemapping.

OS and backend This feature is not backend nor OS specific and could be deployed across most devices Filament supports, including Mobile GL with GLES Compute Shaders, Vulkan with Vulkan Compute shaders, and WebGL2.

[romainguy]

WebGL 2 does not support compute shaders. There was a draft for a WebGL 2 Compute extension but it's been superseded by the WebGPU effort.

Moving some of the post processing to compute, even async compute, will not cut frame times in half on mobile. The bulk of the work remains on the ALUs and texture units.

[Nolram12345]

It's true that WebGL2 does not support compute shaders - for that the current frag/vert option would certainly be a good fallback. My statement that it would cut frame times in half were not specifically about mobile. These types of best-case gains would mainly be visible on Desktop at resolutions of 4K. But the performance advantage is still significant, even at lower resolutions and on mobile. Particularly on the latest devices, which have displays that are in fact at 1440p, fragment throughput and synchronous evaluation becomes a higher bottleneck, especially when rendering more complex scenes. Frag performance currently seems like the biggest bottleneck Filament has in most configurations I've tested.

[pixelflinger]

this is currently not a priority