Mobile Performance

[rogerbraun]

Performance on retina display iPods and iPhone 4 is really bad, because they seem to struggle with anything complex in the fragment shader. Removing the color transform from the fragment shader results in a huge performance boost.

A quick fix would be to make two shader programs, one with color transform, one without, and only use the expensive one if it's actually needed.

Another thing that would be great is drawing in batches like it is implemented in Starling, where everything on the same texture is drawn at the same time, if possible. For reference: http://wiki.starling-framework.org/manual/performance_optimization?s[]=performance#minimize_state_changes.

[rogerbraun]

This was the reason I had to switch to Starling for a mobile project. This probably is not a big problem for better hardware, or even non-iOS devices with similar hardware, but I guess a lot of people might be targeting these devices.

[arkeus]

I'll look into the performance issues. Having a simpler shader that you can alternately choose seems like a good idea.

As for drawing in batches, if you have multiple sprites using the same texture, it's suggested you use AxCloud, which will do exactly that. Everything in a single AxCloud must use the same texture, but it will batch the drawing into a single call. You can also disable stuff such as animations or color transforms in an AxCloud to remove extra cpu processing (but it won't change the shader). I prefer having the user determine what is batched in this method, rather than doing magic behind the scenes, as it makes it easier for you to determine exactly what is happening.

[rogerbraun]

Yes, AxClouds are pretty good, but the "magic" that Starling does is not that magical at all. It just walks the DisplayObject tree and adds batches as long as they use the same texture, which is especially nice when you are using a texture atlas.

For example, to be sure that some sprites that use the same texture are drawn in one call, just add them to a Sprite as children and that's that.

I guess what I'd really like to see is a version of AxCloud with support for a texture atlas.

[arkeus]

The magic that happens though is how it does the drawing. If you're specifically adding things with the same texture to a group yourself, you're sure that everything is drawn in the order you specify, and that you know it will be a batch call. And if you have an object you need to have drawn separately (even if it's the same texture), because you want to override its draw functionality to provide extra, you don't have to worry about it trying to batch it, etc. Maybe I should look into starling more, but from the sound of it it doesn't seem super intuitive to me.

As for texture atlases, perhaps you can explain it to me:

My understanding is that you define a texture atlas to have a particular graphic, and you add all the various images and animations to it (I think in XML)? In Axel, you simply load the graphic on different sprites, add the animations there (rather than in XML and referring by name). To me it seems like these 2 things will result in the same thing (everything will end up using the same graphic in a single call), without the overhead of having to define the animations in XML (I find it much easier to do define it as a series of frames, though I do see the benefit of being able to declare all the positions explicitly even if it's more work). Is there something I'm missing about how starling handles atlases that I'm not understanding? I really want to actually create something in starling to get a better understanding of how it's done but I just don't have the time currently.

[rogerbraun]

I think the Starlings behavior is really good in this regard. It will try to batch as much as it can, but if you want to batch something explicitly, you can do so, too. I think it's a good default, because it can only make things faster and you can still do manual grouping for batching.

Regarding texture atlases:

In the scenario you are describing, the two approaches are indeed identical. But the way I use it is that I actually put everything in a level (characters, UI, enemies, whatever) on the same texture atlas, which you can easily do with TexturePacker from http://www.codeandweb.com/, for example. Now, all the things on the screen are using the same texture, so the whole screen will be drawn in one call. I guess one could emulate this right now by using different animations for different sprites in an AxCloud, but this will not work if your sprites have different sizes.

Then again, this is only a problem on oldish mobile devices. I never had any problem on the desktop or in the browser.

[arkeus]

But I think that's the exact same thing as if you:

1. Used texture packer the same way
2. Have a player class, load(TEXTURE, width, height), add your animations
3. Have a coin class, load(TEXTURE, coinwidth, coinheight), add your animations
4. Have an enemy class, load(ENEMY, enemywidth, enemyheight)
5. Add them all to an AxCloud (or nested axclouds if you want to keep them logically separate for things like collision)

At that point, they are all using the exact same texture, their widths/heights are all different, and they're all drawn in a single call. To me it seems like the end result is still the same, the approach is just different (it's a flixel-like approach to batching).

Edit: Actually I'm unsure of the nested clouds off the top of my head! But you could add them also to groups to keep them logically separate if needed.

[rogerbraun]

You are right, I forgot that Axel caches textures. But it will still be hard if your coin sizes and player sizes are different. For example, a player image has size 13, a coin image has size 7, both have 3 animation phases. Now if I put the player images first, the fourth image will be at position 39. But the next multiple of 7 is 42, so I will have to add padding. This could become really tedious without any automatic tool, and it wastes space, especially if you have images of wildly different sizes.

[arkeus]

Okay, I can definitely see that as a benefit of being able to define them specifically, and that would be somewhere where defining absolute positioning would be great.

I remember when I was thinking this through before, I liked defining atlases through code rather than XML, since it would be much more compact, and less extra data (and less likelihood for typos as it would be syntax checked at compile time). Do you have any reasons off the top of your why XML would be preferable?

[rogerbraun]

I don't like XML, either, and I would never think about writing it by hand. Still I can come up with some reasons for XML in this case:

1. There are already tools to make this easy, TexturePacker for example. It can even automatically trim parts with alpha only, which can reduce texture size greatly.
2. You can split your workflow between artists and programmers more easily. If you need a texture as a programmer you just load the XML. If you need to adjust sizes as an artist, just do so an re-run TexturePacker.

I know this sounds like an ad for TexturePacker, sorry ;-)

Maybe another description format like JSON would be easier to implement, to read and to write (at least by hand).

[rogerbraun]

BTW, Axel is a great framework! I really liked working with it and would have kept doing it if I could have found a way to fix the performance on the iPhone 4.

[arkeus]

Awesome, thanks for the help! It sounds like supporting XML is good for use with automated tools (I think I read that Flash CS6 can also export animations as texture atlases and those are probably XML too). So I'll probably need to support loading from XML, but perhaps also allow you to manually do it in code for simpler tasks you want to do quickly. Your input has been extremely helpful in letting me know which features I definitely need to support, even if it's not something I use. :)