Allow using higher-res images in UI for scaling factors > 1.0, so UI does not look blurry/pixelated

[inodentry]

What problem does this solve or what need does it fill?

It is currently very difficult to make a UI (with images, not just text) that looks crisp on HIDPI displays.

Bevy aims to transparently handle scaling (from window scaling factor + UiScale). The out-of-the-box behavior is that text sizes are increased (correctly, resulting in crisp text rendered at high resolution), but images are upscaled (making them blurry).

I would like the icons in my game UI to look crisp.

For example, I could create different variants of my assets: small-size (to use with scale = 1.0) and large-size (to use with scale > 1.0, designed to look best on 2.0).

There seems to be no way to use these different images in Bevy, in accordance with the scaling factor, unless I write custom system that tries to undo bevy's built-in scaling and choose the correct image handle.

Support for this sort of thing should be built into the engine.

Or, at the very least, support for un-scaled images (as per #7349) would make it easier to implement it myself via a custom system (just select the correct image handle based on scale), without me having to undo or workaround bevy's automatic scaling of all images.

What solution would you like?

I don't know what exactly would be best, we should discuss our options.

What alternative(s) have you considered?

Introduce some special asset type for UI Icon Sets

Perhaps a new asset type for an "icon set", which references multiple images. It wouldn't be without precedent: we currently have TextureAtlas, which is another special bevy-internal asset type on top of Image, to help with using images correctly in a specific scenario / use case. We could change UiImage (or introduce a new component type, say UiIcon) to use a Handle<IconSet> instead of Handle<Image>. Users could create such IconSet assets with similar workflows/apis as for TextureAtlas assets .

With some cleverness, this could possibly be made to allow the engine to only load the variant for the size that will actually be used.

Extend UiImage component to support multiple asset handles

UiImage could be extended to allow specifying multiple asset handles for different scaling factors. They should be optional, and if any are missing, bevy can use the best one available and scale it. Downside: requires all the images to be loaded, even if they are not going to be used (a typical user would use the app on the same display for the entire lifetime of the app; scaling factor typically only changes if the user moves the window to a different monitor or something like that).

Implement it via TextureAtlas

If we want to use just a single Image asset/handle, we could make users put the different variants into one "spritesheet"-style image. This would feel like a very "game engine"-esque solution. ;)

It would save on the complexity of having to deal with loading multiple separate assets and switching between different asset handles.

We could then use the existing TextureAtlas asset type to represent the icon set. Each size would be a sprite index. Bevy could have an internal system that looks at the dimensions of the various rects in the atlas, and picks an appropriate index automatically.

Creating texture atlases from arbitrary differently-sized rectangles is not very ergonomic in Bevy, though. Some UX improvements might be needed in this area.

Use mipmaps in the Image asset

Bad idea. Including for completeness. Mipmaps are always forced to be exactly 1/2, 1/4, 1/8, ... the size of the original image. They are intended for GPU hardware efficiency and texture filtering quality in 3D, not pretty 2D icons and sprites. Besides, it would make the asset authoring workflow for UI images very annoying.

Additional context

See how, for example, pretty much every GUI desktop environment / operating system ever made uses special icon files to support this. Application (and other) icon files contain multiple images/variants for different sizes. The software will pick the best one, depending on where it is displayed.

Also note that, for UI purposes, contrary to common "programmer logic", it is often better to use raster images with different predefined sizes, instead of vector art, which is naturally scalable to any size. This is because, from an artist's point of view, the artist has the ability to design each size individually, to make sure it looks good. Vector art can be conveniently rendered at any resolution, but for icons and UI elements, is likely to not look good at all sizes. This is why most software uses raster icons to this day.

[alice-i-cecile]

Hmm, this reminds me a lot of automatic mip-map generation and selection. Like you said, they can't be used directly, but some of the design challenges remain the same.

I would start with a solution based on #7349. Long-term, I think my preference is for a solution based on IconSet, which likely shares code with TextureAtlas. There are important structural constraints to enforce, but the core idea is the same, so we should probably just wrap it with a nice API.

[ickshonpe]

I planned to add this and a bunch of other features to the image widget and wrote some prototypes etc a year ago but gave up because I couldn't get the first UI image rotation merged. I've got a bunch of other image and color changes I want to upstream soon and could easily add support for multiple image scaling to the api.

[inodentry]

@ickshonpe Let's hope you have better luck with the PR review process this time. I remember your PR on image rotation. I was strongly in favor of it. It's such a shame that it was dragged down by bikeshedding and unproductive discussion and never went anywhere... :(