Billboard sprites (Sprites in a 3D camera)

[rodya-mirov]

What problem does this solve or what need does it fill?

A lot of PS1-era games had a mix of 3D scenes and sprite graphics (pixel art or similar) -- a classic was Final Fantasy Tactics but there are plenty of others. You can have simple 3D models (which are basically just huge voxels with pixel art textures) then your more high-res entities, like people, are animated sprites which exist in the world.

This gives you some advantages of a 3D graphics layout -- basic camera control, rotations, natural scaling, etc. -- without having to actually make complex 3D models for your characters, and while retaining that art style.

What solution would you like?

A billboard sprite bundle; so it's visible at the "normal" location like another 3D object, but it's just a mesh (?) that has one face, and it always faces the camera.

Or maybe it makes sense to somehow just have "billboard" be a component on an entity, so every frame it just resets to face the camera, and some other method could display how to make pixel art work in 3D (there is no obvious way to use a SpriteBundle in a 3D camera that I could make work).

What alternative(s) have you considered?

It's probably possible to implement this manually -- every frame, after update but before rendering, to find all the billboards and the relevant camera, and rotate their faces to meet the camera perfectly. But that seems hard, for a game developer who doesn't really understand 3D graphics and coordinate transforms (me). Seems like something that could be built into bevy, or set up as a plugin (?).

Additional context

A sample screenshot of FFT which I found on the internet and is probably not kosher to truly repost here, but which shows the value of the approach: https://www.cavesofnarshe.com/fft/screenshots/images/1-38.jpg

This is supported in monogame (the successor of XNA): https://liam256.github.io/blog.io/2017/01/05/Monogame-Billboarding/

[rodya-mirov]

Hmm, I'm seeing a rejected (possibly revivable) PR attempting to solve this problem https://github.com/bevyengine/bevy/pull/2814

[alice-i-cecile]

Yep, that was an earlier attempt at this! It's desired, and should be feasible.

There's actually active chatter about this occurring on Discord, see the #billboards thread in #rendering-dev! @Sheepyhead and @inodentry were the main folks investigating I believe.

[rodya-mirov]

That's great! I'll check out the discord channel at some point to get a sense, I guess.

I'm unfamiliar with Bevy process, though. Assuming they get everything nice and settled and implemented, is this something I'd need to wait for Bevy 0.7 for? Or is this something that can (will?) be released as a plug-in without needing a major Bevy release along with it?

[alice-i-cecile]

I'm unfamiliar with Bevy process, though. Assuming they get everything nice and settled and implemented, is this something I'd need to wait for Bevy 0.7 for? Or is this something that can (will?) be released as a plug-in without needing a major Bevy release along with it?

Right now, the official Bevy engine is only shipping this repo; we're not splitting out code into external plugins. And while we're about to have our first minor release (0.6.1), something like this is too large and risky. The next release should be mid-April or so though, we've moved to a 3 month train release strategy.

However, this will probably be implementable without needing to touch the engine internals. As a result, you should be able to steal code from the PR and package it up into an external plugin nicely.

[stijnfrishert]

Ooohohoho, seconding this, exactly what I'm looking for! The sprites don't need to rotate towards the camera for my use case, but I'd be happy to help out :)

[stijnfrishert]

Seeing the old PR https://github.com/bevyengine/bevy/pull/2814:

I agree 2d and 3d sprites will likely even diverge more in the future, so I agree with the reason it wasn't merged.

Do we still think it's a good idea to split up TextureAtlasSprite into TextureAtlasEntry and Sprite? We can then at least reuse TextureAtlasEntry together with whatever ends up becoming the "3d sprite component".

In fact, maybe we will end up with multiple ways of rendering atlas entries even in 2d.

Would it be worth splitting that off in a separate PR to do first?

[VictorKoenders]

Billboards are one of the things that Text3dBundle should solve: https://github.com/bevyengine/bevy/pull/7426

[alice-i-cecile]

I've found a nice external crate for this for now: https://github.com/FraserLee/bevy_sprite3d

[kulkalkul]

Right now there are 3 usecases for this:

• Rendering far away objects as billboards or artistically using billboards: single image
• Rendering overhead text like mmorpg games does: text
• Rendering impostors: multiple images depending on view

Right now I have an implementation in https://github.com/kulkalkul/bevy_mod_billboard that does the first two. It uses a simple shader. For the first case resetting their rotations in their transform matrices is enough. Text is a little more involved, requires building of a mesh from text data and font atlas. Change detection on Text helps a lot in this case, similar to how 2D text works.

Hard part would be batching it. Support for multiple fonts complicates the batching. I'm not familiar with bevy's new batching, so I don't know how it would perform with it.

And there are some useful features that can be easily implemented on top like enabling/disabling depth culling, axis locking, full rotation locking (basic worldspace text).

For a bevy implementation, what are other requirements?

[alice-i-cecile]

For an initial Bevy implementation, I'm most concerned about:

• a straightforward and clear API
• good docs
• well-isolated
• low code complexity

High-performance and fancy features can come later :)