Add a DigitDisplay UI element

[pushfoo]

Enhancement request:

What should be added/changed?

A DigitDisplay UI class that allows the score to be set without using a text label. It would support custom bitmap fonts to display scores or time remaining fancily.

What would it help with?

tl;dr: Things like the score, coin counter, and time remaining display in Super Mario Bros. It would be much faster than the current text labels.

Right now, text UI elements allocate three new textures each time the text is changed. This means that using a text label to display a score can thrash texture allocation when it is updated rapidly. At least two people (including myself) have been frustrated by this unexpected behavior from text labels when attempting to add time remaining or score displays to games.

Possible use cases include:

• overriding a conversion method to convert raw seconds to a timer displaying f"{new_value // 60}:{new_value % 60}" in bitmap sprite form
• displays in puzzle exploration games like Myst using unconventional digit systems (base 7, cistercian numerals, etc)
  • an emulator showing integers as hex or octal

Why do this now instead of fixing UITextLabel?

My understanding of the discord discussion around text labels is that the text rendering update has been in progress for a while and has unresolved prerequisites. A small, usable working digit display class that doesn't confuse new users with performance issues can be implementable faster than the text rendering update gets merged. Once we settle on an API for a digit display, the implementation can be replaced once the text rendering refresh is merged.

[einarf]

Can't this be easily extended to a bitmapped font thing supporting latin1? Then it will probably have a longer life span.

If we end up making a custom renderer for UI this will be depreacted pretty fast. Something independent of the UI would be much easier.

[pushfoo]

Can't this be easily extended to a bitmapped font thing supporting latin1? Then it will probably have a longer life span.

I apologize if I'm misunderstanding, but I think I might implicitly intend to support bitmap characters from latin1. This ticket's scope was intended to be an initial implementation for a digit display with an integer current_value property. For now, it would be backed by a list of sprite objects that grows longer as the score text increases in length. Sprite objects would be used to draw each digit after setting their textures on updates.

This is part of why I was initially pushing for merging a stateful sprite class. Loading frames for each possible glyph into a state table would enable animated font effects such as line boil.

For clarification, here's some example psuedocode:

class CharacterSprite(StatefulSprite):
   """ definition ommitted here """
   @property
   def current_value(self) -> int:
       return self._current_number

   @current_value.setter
   def current_value(self, new_value):

       if new_value == self._current_value:
           return

       self._current_value = new_value
       # update characters whenever a new value is set
       self._update_characters()

class LineBoilCharacter(CharacterSprite)
    # map individual characters to line boil animation for it
    animation_table: Dict[str, List[KeyFrame]]  = load_animated_font("assets/line_boil_font")

    def __init__(self, initial_state=" "):
        """ omitted """

# this will be a 0 that wiggles when shown on screen
wiggly_0 = LineBoilCharacter(initial_state="0")

Regardless of whether we use stateful sprites to back this, I intended to have a fallback behavior of rendering each digit from TTF if no bitmap table was provided.

The internal implementation could be replaced with something more efficient in the future.

If we end up making a custom renderer for UI this will be depreacted pretty fast. Something independent of the UI would be much easier.

I still think it may be worth encapsulating, especially if there's a mode argument/property or overridable method for changing how self.current_value gets displayed. Use cases I can see for this include:

• overriding a conversion method to convert raw seconds to a timer displaying f"{new_value // 60}:{new_value % 60}" in bitmap sprite form
• displays in puzzle exploration games like Myst using unconventional digit systems (base 7, cistercian numerals, etc)
  • an emulator showing integers as hex or octal

Do you have an opinion on whether there should be mode arguments vs method overrides in base classes (HexDisplay, OctalDisplay, TimerDisplay)? I'm leaning toward subclassing and method overrides.

[eruvanos]

Something like this would be enough I guess, kind of easy to use, without any UI usage:

Compatible with the ui_development branch.

https://pastebin.com/SnhrYbw2

from arcade import SpriteList, Texture, Sprite, start_render
from arcade.examples.perf_test.stress_test_draw_moving_arcade import FPSCounter
from arcade.gui import text_utils

class ScoreDisplay:
    def __init__(self,
                 value: int,
                 format_string='{value}',
                 start_x=0,
                 start_y=0
                 ):
        self.start_x = start_x
        self.start_y = start_y
        self.format_string = format_string

        self._tex_cache = {}
        self._sprites = SpriteList()

        self.font_name = 'arial'
        self.font_size = 16
        self.font_color = 0, 0, 0

        self._value = 0
        self.value = value

    def draw(self):
        self._sprites.draw()

    @property
    def value(self):
        return self._value

    @value.setter
    def value(self, value):
        if self._value == value:
            return

        self._value = value

        start_x = self.start_x
        start_y = self.start_y
        symbols = self.format_string.format(value=self._value)

        self._sprites = SpriteList()
        for symbol in symbols:
            sprite = Sprite()
            sprite.texture = self._get_tex(symbol)
            sprite.left = start_x
            sprite.top = start_y

            self._sprites.append(
                sprite
            )

            start_x += sprite.width

    def _get_tex(self, symbol) -> Texture:
        if symbol not in self._tex_cache:
            image = text_utils.create_raw_text_image(
                text=symbol,
                font_name=self.font_name,
                font_size=self.font_size,
                font_color=self.font_color,
            )
            self._tex_cache[symbol] = Texture(symbol, image, hit_box_algorithm='None')

        return self._tex_cache[symbol]

if __name__ == '__main__':
    from arcade import run, View, Window, set_background_color

    window = Window()

    class MyView(View):
        def __init__(self):
            super().__init__()
            self.counter = 0
            self.score = ScoreDisplay(
                0,
                format_string='Running for {value:.2f} seconds',
                start_x=300,
                start_y=300
            )
            self.fps = FPSCounter()

        def on_show_view(self):
            set_background_color((255, 255, 255))

        def update(self, delta_time: float):
            self.score.value += delta_time
            self.fps.tick()

        def on_key_press(self, symbol: int, modifiers: int):
            print(self.fps.get_fps())

        def on_draw(self):
            start_render()
            self.score.draw()

    view = MyView()

    window.show_view(view)
    run()

[pushfoo]

To recap my notes from @eruvanos and I talking earlier + some of my conclusions afterward:

• This would be best based off of the new UI element branch
• UIBoxLayout in horizontal mode is probably the best base class for the container
• It can be implemented without merging an implementation of stateful sprites, but could rely on one in the future if we add them to enable support for animated fonts
• an internal sprite list can be used for the implementation as in the rough sketch that @eruvanos posted above
• character sprites should have their hitbox algorithm turned off and use the texture sizes to set their dimensions
• a baseclass mutable display can take a format string as an argument and gets used in a text conversion method
• subclasses of the mutable display can either pass a new format string to the superconstructor or override the conversion method
• subclasses might require a specific type for the value set and raise TypeError on mismatches

I'll start trying to implement this shortly. Is there anything I missed?

[eruvanos]

848 provides a workaround for the OutOfMemory and decreasing performance for the maintenance branch.

Currently @einarf is working on a different SpriteList implementation, using TextureAtlas. This will help with the performance in general.

Still, the performance of cached textures per glyph is way better, than generating new textures on changing texts.

[einarf]

Made a separate "Fix waiting for release" issue here #849

[pushfoo]

It might be a good thing that I got distracted by other tickets and some responsibilities offline. Regardless of which way we back UI elements, I still think it's a good idea to have typed displays that accept only certain kinds of data. As to how we back them...

@eruvanos I checked out the maintenance branch including @einarf 's latest fix (mentioned by #849 ). I only tested it with a single UI label updated each frame, but it looks performance has improved drastically. Over the span of 60+ minutes, memory usage rapidly peaked at 63MB, oscillated for 10 minutes near the peak, and then followed an asymptotic-seeming curve down to around 17MB where it's sat before I stopped the program. This is very good in comparison to past behavior!

Still, the performance of cached textures per glyph is way better, than generating new textures on changing texts.

We might be able to get the best of both worlds. From what @einarf told me in chat (we have offscreen drawing), it might be worth abstracting packing of regions to be generic and use it for both packing UI layout elements and rendering fonts to an offscreen texture. That way, we can use it to do layout of both bitmap fonts from file and rasterized TTF fonts. The bitmap fonts area distributed in image files that could be loaded as an atlas anyway: (example from opengameart.org)

Using efificient texture atlases rather than the Dict[str, Texture] that started writing would probably be better. We could even make a FontAtlas subclass for TextureAtlas. I see the following benefits to this approach:

1. Removes the overhead of manipulating arcade.Sprite object to set their texture attributes from a Dict[str, arcade.Texture]
2. Supports both bitmapped fonts and rasterized versions of TTF / system fonts
3. No hideous UIElement and UIBoxLayout multiclassing (This assumes we don't need support for vertically oriented text)

If all of this seems reasonable, I can scrap rendering fonts to Dict[str, Texture] I already wrote or rework to apply to TextureAtlases, and then finish this ticket based on einarf's changes once they are merged.

[einarf]

it might be worth abstracting packing of regions to be generic and use it for both packing UI layout elements and rendering fonts to an offscreen texture. That way, we can use it to do layout of both bitmap fonts from file and rasterized TTF fonts.

This is pretty much the "next step" after atlases are in. The text renderer in arcade will no longer create sprites and textures. It will simply just render the text to the a framebuffer (screen or offscreen). The UI currenty have its own system for generating text with pillow, so it doesn't suffer from the caching of arcade.draw_text()

The UI can chose to use this new text redering instead of using PIL images. It's probably a good idea, but one thing at a time! 😄 There might be issues with rendering order and whatnot. Time will show. If the UI can just draw the text every frame that also works fine.

For now the Dict[str, Texture] setup is all good. It will be efficient enough. Once we have atlases we can also cache text glyphs and make text redering super fast without the use of textures.

For the UI class design I have not much to contribute with atm.

[eruvanos]

To wrap it up/Suggestion:

We wait for the atlas, which will solve a lot of performance issues for us, after that, we have a look and improve UILabel, and maybe derive specific display classes from it. The magic around fonts etc will be handled in UILabel.

Right?

Maybe at some point in time it is worth to think about scrolling text areas :D

[einarf]

Two steps 😄 1) Texture Atlases + optimize things to use them efficiently 2) Revamp text rendering completely + explore how it can be used

[pushfoo]

I think some of the changes proposed earlier have been made. Can we make this into a right-aligned subclass that only takes ints?

[einarf]

It this even still relevant? Text objects or spritesheet with numbers should do the job. Using the gui just to display some numbers I think is very overkill.

[pushfoo]

is this even still relevant? Text objects or spritesheet with numbers should do the job.

Agreed!

Using the gui just to display some numbers I think is very overkill.

Not in all cases, but I'm still closing the issue for multiple reasons:

1. The improvements in Arcade's text rendering since filing this issue make it obsolete
2. Bitmap font support is orthogonal to displaying numbers with acceptable refresh rate and should probably be implemented in pyglet first
3. The Kenney fonts we bundle approximate pixel fonts acceptably for prototyping and game jams