Proposal for an "immediate" mesh rendering/vertex submission API

[slime73]

Original report by Max Cahill (Bitbucket: geti, GitHub: geti).

---

An immediate mode vertex submission API would provide:

• a low friction way to submit arbitrary textured geometry
• a very versatile primitive for building other userland rendering APIs
• a way to get started experimenting with vertex attributes in shaders
• a "lower level" way of interacting with the rendering API without managing Mesh objects

This was discussed briefly in discord; slime seemed possibly interested and it was suggested that it be further discussed on bitbucket, so here it is.

Minimum Spec:

#!lua

love.graphics.vertices(
  optional_texture,
  vertex_table_or_data,
  primitive_mode,
  optional_indices
)

Basically a "textured tri" version of the existing polygon API, with the ability to submit positions, uvs, and colours directly. The vertex format would match the existing "built in" mesh format, with the same defaults.

The primitive mode may require flushing the batch; the most useful mode would be "triangles" and could be provided as the only option at a pinch.

Optional user submitted indices (relative to the submitted geometry, not the whole batch) would be ideal, to reduce the amount of data to be transferred and kept up to date for redundant geometry.

Possible Spec Extensions:

#!lua

love.graphics.setVertexFormat(
  same args as mesh:newMesh vertexformat
)

The ability to set a custom vertex format would allow users to experiment with custom vertex formats and shaders in a lower-friction environment than with persistent mesh objects, and avoid "reusing" existing attributes like colour components.

Alternatively, an enum could at least be used to set just from the "existing" batching vertex formats (and those could maybe be extended a little) to avoid extensive reworking of the batching system.

#!lua

love.graphics.setBatchSize(
  size_enum
)

The ability to set the batch index size (to 8, 16, 32 bit at least) would allow people to trade off larger or smaller batches for cache bandwidth; 8 bit indices could be used for rendering a lot of heterogeneous sprites, 32 bit for huge meshes/high definition lines/etc. This would either require draw-splitting or "accepting" the current truncation of large draws.

Apparently 32 bit indices are not supported on GL ES platforms - 16 bit is a sane default anyway, and a warning about this could be provided in the documentation.

Things this proposal is not:

This is not intended as a replacement for the Mesh API. The mesh API would still be a significant optimisation for most cases of static geometry.

This is not intended as a replacement for submitting simple textured quads or untextured primitives.

[1bardesign]

Just to note, my github is now @1bardesign, not @geti 👍

[raidho36]

You can just write a wrapper function that does this.

[1bardesign]

Sure, you can "just" write a wrapper function for batched geometry using a mesh, but as a user of love you can't easily do the following:

• continue to batch with existing compatible geometry from other lg calls
• transform verts on the cpu across transformation stack mutations without repeated copying of the transform
• hook the transformation stack mutations without monkeypatching all the lg transformation functions,
• integrate with lg.present to auto-flush without monkeypatching it, alternatives would be either useing a renderTo(f) style api or manual batch flushes, neither give the same user experience
• utilise the already-extant streaming vertex buffers used for batching on the love side to save memory

I feel that the requirement for monkeypatching so much core lg functionality to achieve anything close to this proposal makes it pretty clear that it would be better accomplished from inside love, with access to all the internal data structures.

Further, it's pretty clear that to implement something like this in user space, you'd end up re-implementing all of the batching machinery, which already exists inside of love.

[raidho36]

So basically, you wish that SpriteBatch would accept meshes, not just quads. That's actually perfectly reasonable and not at all difficult to implement. With that in mind, the autobatching mechanism should also add meshes to current batch when possible (I don't know if it currently does but I assume that it doesn't), thereby satisfying your concerns.

What you originally described had virtually nothing to do with batching and it was implied that it was somewhat a debug kind of functionality, all which was just a wrapper over existing functions - there was no reason to implement any of that.

[1bardesign]

To clarify, this proposal is not really intended to be on top of a SpriteBatch object, but to complement the existing immediate drawing API, nor is my concern that meshes themselves aren't batched.

The batching system transforms vertices before submitting them to the gpu as they are assumed to be streamed - meshes do not (and spritebatches do not, iirc). Further, changing that behaviour would be unhelpful as you would lose the ability to transform a static mesh or spritebatch (think terrain patch, tilemap, vegetation) around using the transformation stack without re-uploading verts.

This proposal requires:

• a way to submit arbitrary vertex geometry to the built in batching system, with immediate-style calls.
• as an extension, allow the batching system to accept non-builtin vertex formats, for streaming geometry with custom attributes to shaders without needing a retained mesh in "userland".

If SpriteBatch would be extended to accept arbitrary geometry too (perhaps GeometryBatch at that point?) that would be welcome, of course.

Sidenote: the string "batch" appears 7 times in the initial proposal, so I'm not sure how it quite had nothing to do with batching :wink: It also never mentioned being debug functionality.

This is the heart of the proposal:

An immediate mode vertex submission API would provide:

• a low friction way to submit arbitrary textured geometry

[raidho36]

You mentioned batches but I don't believe it actually has to do with batches. Does the following script implements your idea?

love.graphics.setVertexFormat = function ( vertexformat )
    if not vertexformat then
        love.graphics._vertexformat = nil
    else
        local success, errmsg = pcall ( love.graphics.newMesh, vertexformat, { { } }, 'fan', 'dynamic' )
        if not success then
            error ( errmsg )
        end
        love.graphics._vertexformat = deepcopy ( vertexformat )
    end
end

love.graphics.getVertexFormat = function ( )
    return deepcopy ( love.graphics._vertexformat )
end

love.graphics.vertices = function ( texture, vertices, mode )
    local mesh
    if love.graphics._vertexformat then
        mesh = love.graphics.newMesh ( love.graphics._vertexformat, vertices, mode, 'dynamic' )
    else
        mesh = love.graphics.newMesh ( vertices, mode, 'dynamic' )
    end

    if texture then
        mesh:setTexture ( texture )
    end
    love.graphics.draw ( mesh )
end

I don't exactly understand your concern with transformations and batching: you can't really batch multiple large meshes because you have to transform every single vertex (since the entire mesh shares a global transformation) and that's not really feasible, particularly not in immediate mode, so naturally you default to not transforming anything, and by extension to breaking the batch - as it currently is. Not to mention, shall you use custom vertex format or a texture for your polygon you'll have to do a shader and/or texture switch, which will break the batch regardless - and doing so seems to be the idea.

[pablomayobre]

Batch (verb): arrange (things) in sets or groups.

LÖVE 11.0+ (since 0d2e08baff01f0a103e5548d1636557887e40205) Auto-batches draw calls that use the same resources, for example love.graphics.circle and love.graphics.rectangle don't need to be two separate draw calls, they can be grouped as one. All the vertices are put in the same mesh, and once they need to be draw to the screen they will be rendered in the same draw call.

The same applies to rendering the same image multiple times (with or without different Quads), the draw calls can be batched into a single one. To do this, LÖVE uses an internal "SpriteBatch".

But this is not really the same as a SpriteBatch exposed to the Lua side, it has some other logics and checks for state changes during love.draw that may require the batch to be flushed and rendered to screen and it also transforms the vertices on the CPU and some other things.

This batch logic could be used with Mesh data, in addition to the Quads I mentioned before.

The idea proposed here is that you could feed data to the internal mesh used by LÖVE Auto-batching mechanism, and as long as the state remains the same (no BlendMode change, no Shader change, no Texture change, etc) you could continue grouping it.

Once the contents are flushed it would all belong to a single draw call.

Your example doesn't do this, each call to love.graphics.vertices becomes one draw call. There is no batching

I hope this clarifies the idea so that the issue can be discussed properly.

Some notes to clarify some of your doubts: When feeding a large amount of data you would split the draw call because the Buffer would be full. Different vertex formats and Textures would break the batch, but that's the same as it is right now when rendering quads, so no issues there.

[raidho36]

This is a great answer to a question that nobody asked, thank you. To paraphrase: why concern with batching if it cannot be achieved under intended use case?

[1bardesign]

Intended use case of streaming arbitrary textured geometry can be batched, just as textured quads can be batched currently with love.graphics.draw(t, q, x, y).

All that is required to do so, in fact, is a way of submitting geometry to exactly the same code as is already batching the aforementioned call, which is what this issue proposes.

[pablomayobre]

It can be achieved.

It's of course limited, you won't always benefit from batching.

You need to fill the requirements:

• Small amount of Geometry (so it doesn't exceed the Buffer limits)
• Same Vertex Format
• Same Texture

As long as you have this, you'll benefit of the batching mechanism.

If you are not interested in the proposal I would suggest to just leave it to the devs to decide.

[raidho36]

I don't find anything contentious about adding ability to draw textured polygons (with is a small step up from existing functionality of drawing untextured polygons) and throwing in vertex format definition while we're at it - it should exist and I believe it should've been like this to begin with. Your original post made it sound like a narrow scope duplicate feature, instead. Perhaps if you phrased the original post like this the entire argument could've been avoided.