Allow copying resources.

[seisatsu]

Sometimes if we are going to be doing operations on a resource, we want multiple copies of the resource. If all of the places it's being used are just references, editing it will change them all. This is not desirable for things like lights whose colors can be changed, because it is not possible to load the same light twice since the filename is the same; ResourceManager just gives us the first copy from the cache which has already been operated on. This is why caching was disabled for lights. Instead, we should be able to spawn a copy of any resource, especially FileType resources.

[seisatsu]

With font files, we need to load the file once for each pt size so the glyphs can be rendered. We internally append a colon and a pt size number to the end of the filename in the cache to differentiate them. We could use this for copies of resources as well. Certain resources, like FontFiles, do not need to be copied and already identify by pt size, so they shouldn't be copyable. But lights, and possibly sound effects, and I'm sure some other things will need to be copied pretty often.

[pmer]

Makes sense for images.

Haven't heard of sound effects being copied before, though. Can you describe this? I'm asking because maybe we can skip it (for now?).

[pmer]

Editing images can be an expensive operation depending on how we support it.

What type of API do we want to support? The more flexible we are (the more control we give to game authors), the slower it'll be due to the hardware architecture of modern graphics processors.

• Giving control over setting the color of every individual pixel is slow since this is done on the CPU & requires an upload to the GPU afterward. Maybe we can get away with this, but maybe not. We can investigate.
• SDL has built-in commands for multiplying a color in to a RenderCopy(). See SDL_SetTextureColorMod. We can change the color mod every frame & doesn't require copying a texture at all so can use the original image instance. This does not involve the CPU. It's pretty limited, though.
• We might also be able to do slightly more complex image modifications by creating a texture, setting it as the SDL render target and drawing to it. This will keep all the drawing on the GPU & won't require a texture transfer.

It's easier to provide limited functionality now and expand it with later version of Driftwood than it is to provide more functionality now and restrict it with future versions.

[pmer]

Just noticed we already use SDL's color mod.

If that's all we need at this point I think it might be possible to use it without copying textures... Not sure though.

[seisatsu]

For sound files, whether it's playing or not and other settings are a property of SDL's handle on the sound file. Asking to play the same sound effect multiple times concurrently will give us a reference to the same place, so you can't limit operations to just one instance of it.

[pmer]

Asking the same sound chunk to play multiple times concurrently seems to work correctly.

[seisatsu]

In that case I may be mistaken. Since you've self-assigned it, try to break it.

[pmer]

Can't break it.

If it turns out broken later because of something I didn't catch, feel free to open an issue.

I'm going to make color mods work. I believe after that we can mark this issue resolved!

[pmer]

Looks like SDL_mixer can only play 8 sounds at once.

[seisatsu]

Did you solve copying images?

Currently, the user can only change an image by putting images, widgets, or lights on top of it. I don't think we support any advanced image editing features except the little we do with lights. If the user wants to do more complex stuff currently, we might want to give them a way to request the next frame from the framemanager so they can import PySDL2 in their script and do something with it.

GLSL or other shaders are a stretch goal for this engine. So, we will eventually want very good capabilities for the user to do their own operations on graphics.

[pmer]

Here's an idea to support copying of textures:

When we copy a texture, it is inserted into a LifetimeManager with a unique, randomly generated key. LifetimeManager is like CacheManager but doesn't use a TTL and just destroys objects as soon as they are not used anymore.

[pmer]

Actually LifetimeManager can keep its resources in a set. Its resources don't require a user-made key.

[seisatsu]

I think you can just merge that functionality into CacheManager.

[pmer]

Hey @seisatsu do you think it would be better for Driftwood to work with SDL Surfaces instead of Textures? I just read http://sdl.5483.n7.nabble.com/Proper-way-to-duplicate-a-SDL-Texture-td37434.html.

What they say in that thread also jibes with my own mental model of graphics programming. I'm not 100% certain since I don't know what your plans are for the graphics architecture, but IMO it sounds like you are thinking about images as SDL Surfaces.

[pmer]

BTW working on texture duplication. Will post a PR for it maybe tomorrow.

My previous comment is about the future of Driftwood graphics & your vision for it.

[seisatsu]

It's possible that I originally misunderstood in what cases there is benefit to using textures. I'll look over it.

[pmer]

I will post more info for this discussion & try to give my viewpoint on surfaces/textures tomorrow.

[pmer]

TL;DR:

• Surfaces are simple to use but slow on hardware
• Textures are complex to use but fast on hardware

There is a trade-off between: the API we present to game authors (how complex it is to write games) versus Raspberry Pi (and related systems) performance.

---

Both Surfaces and Textures are a 2D raster grid of pixels, but Surfaces are kept in system RAM while Textures are kept in video RAM.

On modern computers, the CPU can only read & write to system RAM and the GPU can only read & write to video RAM. Historically, this restriction came about because GPUs were mounted on a video card that often came with its own RAM chips. That was the video RAM, and it wasn't exposed to the CPU. Today, a growing number of systems use integrated GPUs that bundle the CPU and GPU on the same silicon die. It is technically possible, therefore, for the CPU on these systems to access video memory and for the GPU to access system memory. But to remain compatible with systems that use actual video cards & have dedicated, separate video RAM, programmers are not allows to take advantage of this technicality.

It is always possible to transfer the contents of a Surface to a Texture. That's how GPUs get all their data in the first place. But it's not always possible to transfer the contents of a Texture to a Surface. Whether you can or not actually depends on which hardware and which driver you're using. This is done for optimization reasons: graphics processors can be designed faster & more efficient if programmers follow the convention of never transferring Textures to Surfaces. And people like speed & low power usage so much, the designers of the hardware just didn't built the capability to even perform this transfer for every type of Texture. (That way you can prevent those pesky programmers from messing up the performance of your processor. Also, it's probably cheaper.)

Okay, so if the CPU isn't allowed to access & modify a Texture, how can we, the programmers, modify textures? We have to use the GPU. And the way that modern GPUs are designed is they have tens (Imagination Technologies & Intel) to hundreds or thousands (AMD & Nvidia) of cores. Shaders are how we program those cores, and as the power and efficiency of those cores increases year after year, the importance of using shaders increases as well.

Before GPUs were created, all graphics were done on the CPU, so all you had were (pre-SDL) Surfaces. Up until the mid-90s, all games were written using Surfaces. SDL 1.2 was behind the times when it came out because it only supported Surfaces & didn't support Textures.

With the use of Surfaces comes a particular way of thinking about graphics. You can pass around a Surface to anybody and they can write whatever they want to it. You have absolute freedom and control since, when you program, you control the CPU and the CPU controls Surfaces.

But when you use Textures you are limited by what your graphics API allows you to ask your GPU to do.

With Surfaces, you can't have very many pixels (hense the pixelated nature of old games) and you can't have a lot of complicated special effects because CPUs don't have enough processing power. The rise of shaders is what has allowed games to reach 720p, 1080p, and 4K resolutions, or to achieve modern lighting.

---

Working with surfaces is extremely easy. Just think of it as a 2D array of pixels that you can do anything with. Bam, done.

Working with textures means we restrict ourselves to the SDL_Render and OpenGL functions to perform any and all graphic operations. (But notice how limited and small the SDL_Render API is.) These functions are basically proxies that will cause the CPU to ask the GPU to do the operations we want. We cannot manually perform any graphics operations by reading or writing pixels directly with the CPU.

---

With Driftwood we have a choice.

Driftwood renders very low resolution frames, so its drawing can be done on the CPU. That is, we can do everything in Surfaces. This will be slow and power-hungry but will be the easiest to program. It also will allow a game author to develop their own special effects using just Python and no shaders.

Our other choice will be to use Textures. We'll have to learn how to talk to the graphics card & might need to learn OpenGL. But Textures will be faster, allowing the Raspberry Pi to achieve higher frame rates, and this will use less power on laptops.

A third option would be to combine Surfaces and Textures to try to get some of the advantages of both. But we have to be mindful of the one-way nature of the Surface to Texture transfer.

[pmer]

Oh, this whole time I was thinking that with textures we couldn't present the final composed frame to game authors so they can draw special effects on it. I was thinking only Surfaces could do that. That's actually where I was coming from when I was asking whether we should use Surfaces throughout the whole program, hahah.

But I just realized that if we create the back buffer with SDL_TEXTUREACCESS_TARGET then it will work, and it looks like we already do that.

[pmer]

It's possible the Surface / Texture discussion might come up again later but I think we can stick with Textures for now.

[seisatsu]

I had been using textures because I read that they were hardware accelerated while surfaces were not. Back then I didn't really understand what that meant. It's probably fine for now.

Eventually we will need textures anyway for things like shaders and environment-aware lighting.

[seisatsu]

Ok, so I looked over it, and basically what I'm doing is performing all graphical copy operations or other built-in operations such as changing alpha or color mod using textures. Operations with textures are accelerated by the GPU, while surfaces are not. Surfaces are only used where needed, but may eventually be needed for non-trivial graphical operations that can't be accelerated.

If you can find a place where using textures actually makes us slower, it should be switched over to use surfaces.

[pmer]

We should find that all graphical operations can be accelerated but some may require the use of OpenGL, which neither of us know much about.

AFAICT textures will never make us slower than surfaces.