Shaders using an image's origin or size tend not to be batched

Zyko0 commented 11 months ago

Operating System

[ ] Windows
[ ] macOS
[ ] Linux
[ ] FreeBSD
[ ] OpenBSD
[ ] Android
[ ] iOS
[ ] Nintendo Switch
[ ] PlayStation 5
[ ] Xbox
[ ] Web Browsers

What feature would you like to be added?

Following a discussion on discord

The graphic commands DrawTrianglesShader and DrawRectShader with different source images and/or on different destination images cannot be batched if they use origin or size information about the images, even if the source and destination textures are the same for several consecutive calls, e.g:

func Fragment(dst vec4, src vec2, color vec4) vec4 {
  origin, size := imageSrc1Origin(), imageSrc1Size()
  dstorigin, dstsize := imageDstOrigin(), imageDstSize()
  // do whatever calculations because you need this (idk why but can happen)
  return imageSrc1At(newSrc)
}

Because the information is stored in two uniforms, that need to be re-uploaded with the new values (https://github.com/hajimehoshi/ebiten/blob/v2.7.0-alpha.2/internal/graphics/shader.go#L77-L81)

Suggestion (but I might be missing some details):

Introduce an integer ID for an image on a texture atlas
Have an UBO per atlas texture (Uniform buffer object) storing [N]vec2 for both origin, size (x2 or a vec4) of the N images (N can be a fixed limit per texture) that can be indexed by the image ID
Populate all ebitengine vertices with two internally reserved attributes (one for src image ID, another for dst image ID) edit: Actually 5 should be reserved for the 4 sources images, and the 1 dst image
Declare two varying int (or an ivec2) on vertex/fragment's base code, and assign them IDs of src, dst images (from vertex attributes) (https://jameshfisher.com/2017/10/19/glsl-varying/) from the vertex shader.
The varying can be accessed from the Fragment shader for imageSrcNOrigin, imageSrcNSize, imageDstOrigin, imageDstSize to find origin and size information from either the srcUBO[id] or the dstUBO[id]

I'm not sure it would work, since it's only theoretical from my point of view, so I might be missing crucial details. And even though there seems to exist counterparts for other graphic drivers, the above idea is opengl-specific in terms of wording at least.

Why is this needed?

Custom shaders that make use of origin or size information from a source or a destination texture, don't make use of atlasing at all, which is unfortunate. There's no way to batch them (e.g: 500 triangles with the above shader would be catastophic).

The usecase is not so uncommon, one could write their own DrawImage with their custom shader making use of origin, size information for a particular effect on all their images, and would be in this situation without control over batching.

An entrypoint or a first proof of concept could be (from the user side) to create its own image as an atlas texture, since .SubImage would contain the origin information, and provide it manually for source and destination in vertex attributes once there's support for more (see https://github.com/hajimehoshi/ebiten/issues/2640). It would need 8 attributes to store srcOriginX, srcOriginY, srcWidth, srcHeight, dstOriginX, dstOriginY, dstWidth, dstHeight, in order to "emulate" the 2 id attributes (for src and dst) that would represent the indices of respective UBOs, containing origin,size information.

Zyko0 commented 10 months ago

Note: I think this is called "instancing"

hajimehoshi commented 1 day ago

I've not come up with a good solution. Let me remove the milestone.

hajimehoshi / ebiten