Resource management refactoring

[billyquith]

This issue follows from on the work in PR #83. GWEN had a number of design problems around resource management. Before launching into refactoring 🔨, discuss! 👍

• Who owns the resources?
• Fonts aren't tidied up properly.
• Resource paths may not be cross-plaform.
• Resource may want to come from different sources. Since the API is embeddable, this needs to be flexible.
• Any other issues?

Solution:

• How can this be made flexible without being overly complicated?
• Current solution added ResourceLoader and ResourcePaths to allow customisation of per-platform paths, and then where/what they are loaded from and managed.
  • What are the problems with this?

[topblast]

How resources are handled is specific to the renderer. However, as you mentioned in #83 -

The idea with the ResourceLoader is that you might pass in a different one depending on your use, so during dev you might load from disk, and then a release build might load from a (compressed, encrypted) package.

The approach without the ResourceLoader limits how resources can be loaded as the required renderer properties are restricted. However, the current implementation requires any derived ResourceLoader to define the same data structure which is enclosed in the renderer.

[billyquith]

Let's talk about the functionality that we want:

• How about defining a ResourceType, so that can have texture and fonts separate. There may also be different types of texture and font. There may be other types of resources, perhaps models, or SVG, later.
• Have a ResourceHandler(?) that can create, render, and destroy the resource.
  • This might be an interface that creates known families of resources with known properties that are implemented in different ways. So TextureResource would be 2D, with size and can be rendered at position. Font has family, size, etc.
• ResourceManager keeps track of what's created, and cleans up. This owns the resources. I guess this could also be the renderer. - Superfluous? Do this in handler? But then all types registered here. Single responsibility.
• ResourceLoader fetches the resource given URL/path. This might be from file, package, generated, etc.

The idea is that this API is embeddable. The renderers provided are examples. Users are free to modify and extend. When embedding we need to be very flexible and set too many "rules" for the client application. Be "componentising" the resources and renderer it allows more how swappable environment, and may encourange sharing between some platforms/renderers (e.g. fonts).

The above may be objects or interfaces. Just thinking about functionality and responsiblities at the moment.

[topblast]

These are all interesting points. My concern is that the additional separation will bring unnecessary complexity especially if the current renderers are examples. Wherever special use cases like package & compressed resources are used, extending the renderers is much simpler than the proposed solution. Additionally, the separation of the resources and the renderers is concerning (thinking of the font/texture data).

[billyquith]

I guess one massive problem is the number of backends to support. I suppose if you keep the design simple there are less variables here. I was hoping to add a bgfx one. I guess another way to go is to adopt an embeddable renderer (like bgfx) and have less supported platforms, where you would use bgfx (or one of the other library APIs) instead.

It would be nice to add more of a command-buffer style rendering API, with more primitive control. I wonder if there could be some overlap with those systems. You might then be able to do more complicated widgets, like charts, and render arbitrary shapes.

[billyquith]

I included #83. There are plenty of other areas that need work, as you can see from the issues. Thanks for the work. 😄

[billyquith]

I noticed a few things after merging the code:

• The renderer textures are looked up everytime a rect is drawn. This is too inefficient. How about storing a user data in the Texture.
• The utf8_to_wchart utility has unreachable code (returns).
• Why is Widen too slow? Have you profiled it?
• The software renderer texture colour sampling is broken.

[topblast]

A command-buffer style renderer is interesting and I do agree that the renderer support is fragmented. This is evident with the difference in features between renderers. bgfx is a great renderer and is used by many. I think the popular rendering APIs like OpenGL* / DirectX can be replaced if bgfx provides a means to render to the given APIs without initializing it inside of bgfx.

I profiled Widen and it observed some performance degradation between it and utf8_to_wchart. This may be caused by the allocation and eventually deallocation when using Widen. I will get started on the render lookups and software rendering issue.

[billyquith]

Another feature requested that makes sense is allowing rendering of different Controls with different skins. At the moment all Controls are 9-section sprites, but if the Control renderer is passed the extents and properties it could render any way it pleases. When Controls are created it would be nice to bind a Control renderer to it, so they can be mashed up. I'm not clear on the design of all this yet as I've been concentrating on the reflection/Ponder stuff, which has been taking some time! The Skin also wants to be data driven, with all those UVs and colours coming from a config file (loaded via reflection).

I have a mind to branch Gwork before your change and release as V1.0. And these more substantial changes would be V2.

[billyquith]

A command-buffer style renderer is interesting

Well it kind of has that now. The renderer is free to do what it likes with the Draw*() commands. Perhaps all it needs is for the draw rect/quad to be made into 2 draw tri calls (or strips). Once you have draw tri it is a lot more flexible, and you can also draw lines.

... I do agree that the renderer support is fragmented. This is evident with the difference in features between renderers.

Its mainly the font support that is inconsistent. Perhaps more thought needs putting into how it might be shared between renderers? If FontStash added surely multiple backends could use it?

[topblast]

Once you have draw tri it is a lot more flexible

Such a function may even morph into SVG support.

One of my plans after #83 was to get to try integrating FontStash into the renderers, seen in this branch fontstash-test. Additional FontStash features could be implemented(like blurry text) to all renderers as well. More is to be discussed on how to approach this. FontStash is a source. It may be as an abstract class or maybe some else entirely.

[billyquith]

integrating FontStash into the renderers,

Just to clarify, it only really needs adding to the renderers that don't already have font solutions. Allegro5, SDL2, SFML2, etc all have solid solutions. GL and software lack full unicode support i think. I guess the way I see it is a reusable font module that throws out glyph rectangles to a renderer. I guess the solutions with font libraries can still use this, but this isn't the priority.

[topblast]

The current implementation of Unicode support in Direct X & the one added to GL & SW in #83 renderers has a limited range of characters(Glyphs). All of these characters are baked to a texture while recording their texture positions (many of these may not be used). This approach requires a large texture from the start and the size of the font may change the size required for the texture. Right now, it is set to create a 2048 width regardless of font size. In Direct X the side is height is reduced if it is not needed. Any changes to the font (size, family, bold, etc.)

static const wchar_t BeginCharacter = L' ';    // First Character of Wide Character Table
static const wchar_t LastCharacter = 0x2FFF;   // Last Character of Wide Character Table; 
// The larger LastCharacter is, the longer it takes to bake the Glyphs. 
static const wchar_t NewLineCharacter = L'\n'; // New Line Character

FontStash approach is to bake only the characters being used to one texture. Regardless of font properties like size, family, etc. In the event that a new glyph is requested (one not on the texture), a free location in the texture is reserved for the new glyph and the local(CPU) copy of the texture is updated, then FontStash prompts the renderer to update the GPU texture.

In both methods, DrawTexturedRect is sufficient for each character. FontStash's method of updating the texture for requested characters is what the renderers lack. A texture containing all 0xFFFF characters of wchar_t is inefficient (I never wait for it to finish bake each texture, but I estimate a few minutes for each font) & impossible on many GPUs due to texture size limitations.