Color Picking Shader

[aevyrie]

For performance, I'd like to render the scene to an off-screen buffer that renders each pixel as an entity encoded into RGBA. Picking is then as simple(😆) as querying this buffer and doing a lookup to return the entity. This would only allow picking for the topmost item (no pick list), and would have limitations with transparency.

A hybrid mode that uses both might be a good solution.

Some input from others on Discord for reference: https://discord.com/channels/691052431525675048/743663924229963868/755912884885913751

[aevyrie]

For reference, here's an article that describes how a picking shader works: https://webglfundamentals.org/webgl/lessons/webgl-picking.html

Here's another good reference that shows how Our Machinery addressed picking through alpha masked objects as well: https://ourmachinery.com/post/borderland-part-2-picking/

[arjo129]

We have been developing an application that uses an older version of bevy_mod_picking here. We recently ran into the need to have a color picking shader which I've prototyped in this branch. I am open to refactoring it and submitting a new backend (we do eventually need to migrate our app).

[aevyrie]

There is an open PR to add a picking shader to bevy, probably landing next release, 0.12. The plan is to make a backend to support this.