Polygon mesh sprites

[pSi-X]

Texture Pack implemented the ability to create polygon mesh sprites: https://www.codeandweb.com/blog/2015/10/01/cocos2d-x-performance-optimization Are there any plans in this Starling 2?

[PrimaryFeather]

In Starling 2, anybody could create an extension that implements this! Thanks to the new Mesh class, it's really simple to create arbitrary meshes like this; all that's required is to parse the data exported from Texture Packer.

However, I'm not sure if it would bring performance benefits. Such optimized Meshes help the GPU, since less pixels need to be drawn. However, it becomes more expensive for the CPU: more vertices need to be transformed and moved around during batching.

The problem: in general, Starling is harder on the CPU than on the GPU; so the result might be slower than using the simple quad-based approach.

It does depend on the game, though. Games with many big, overlapping objects would benefit (i.e. games that are GPU-bound).

If there is enough interest, I can look into this, though, and write a parser for that format. Votes are appreciated! :smile:

[b005t3r]

I'm voting this up. Should be a significant performance boost and significant texture atlases size reduction (transparent areas of some sprites can be reused to put other sprites in there).

And about it being more expensive for the CPU, you can always batch "layers" which don't change and reduce the cost of batching those each frame, but you can't do anything with the unnecessary cost of drawing transparent pixels over and over each frame.

I "almost" added it to my Starling port in Kotlin (which I had to put on hold, because of our game Eclipse), so maybe some bits of my code can be useful for you Daniel:

https://github.com/b005t3r/Kotling

[PrimaryFeather]

That's correct: you'll certainly save texture memory, and reused batches will also profit.

Okay, let's give this a try. As a first step, I'll ask Andreas Löw to create a version that adds the data to the Starling XML format. With that in place, I can try it out in Starling.

Thanks for sharing the code, Łukasz!

[pSi-X]

Thanks guys! This is my dream ^_^

[b005t3r]

BTW, I also created a texture packer exporter for this. It's not complex at all, you can give it a look if you need to:

https://github.com/b005t3r/Kotling/tree/master/core/texturepacker/kotling

BTW, one, quite serious drawback when using this is texture atlases generation time - it's much, much longer than with standard, rectangular sprites. Also currently (and I hope Andreas will add it in the future) rotated or mirrored sprites are not supported, which isn't that bad, because sprites are packed quite efficiently, but there's still room for improvement.

[PrimaryFeather]

I have looked into this in more detail, and I wanted to discuss some things with you before I let Andreas Löw makes the necessary additions to TexturePacker.

There's one problem: we can't just add polygon-support to the TextureAtlas and it will magically work, like when I added support for texture rotation. The TextureAtlas returns Textures, and you typically use those to instantiate Images.

It doesn't make much sense to store polygon data inside the Texture class, though — that information just doesn't belong there, in my opinion. It would also have serious side-effects.

• How to create a SubTexture from such a Texture?
• What if I assign it to, say, a MovieClip, where each frame uses a different texture?
• How can I create an Image instance from such a polygon? Image extends Quad and is thus always supposed to be rectangular. Yes, I could ignore that — but what happens when you set the scale9Grid or tileGrid? I can't implement that with the polygon data.

You see, that's just not feasible. :wink:

Thus, I'm proposing a different solution. What if, with "polygon packing" enabled, TexturePacker would create the following file format instead?

<MeshAtlas imagePath="atlas.png" width="520" height="512">
   <Mesh name="bird-1" width="85" height="71">
     <Positions>84,38, 84,58, 34,71, 16,71, 2,20, 12,1, 47,1</Positions>
     <TexCoords>415,438, 415,458, 365,471, 347,471, 333,420, 343,401, 378,401</TexCoords>
     <Indices>4,5,6, 2,4,6, 2,3,4, 0,1,2, 0,2,6</Indices>
   </Mesh>
   <Mesh name="..." />
   <Mesh name="..." />
 </MeshAtlas>

You might already see where this is going: I'm proposing a new class in Starling, called MeshAtlas. That would work similar to the TextureAtlas class; it just wouldn't return Texture-, but Mesh instances.

We could even let this new class support TextureAtlas XMLs — in this direction, this is not a problem. So in the future, if you want to mix and match atlases, you can do that. You just can't do it with the standard TextureAtlas class.

What do you think? Makes sense / bad idea?

[b005t3r]

And why not add polygon data to Textures? I'd suggest adding a new property to textures or creating a subclass, lets call it TexturePatch, which would also contain vertex data (x,y) for the polygon which can be created using this patch.

Textures already contain data which, according to the logic above, shouldn't be there, like frames - these are purely virtual data.

When I was porting Starling to Kotlin, I went with creating TexturePatches only, nothing in between, but if you want to keep backward compatibility with what's already available, I'd go with adding a simple polygon data to texture.

And Images can support this new format as well (I think that's the whole point of having it) and still be rectangular in nature, I don't see why this would not work.

[PrimaryFeather]

Textures already contain data which, according to the logic above, shouldn't be there, like frames - these are purely virtual data.

Yes, and this created a whole lot of problems, actually.

• E.g. that people have to call Texture.frameWidth to get their original (untrimmed) size. I guess 90% of all Starling users are surprised by this when they encounter it the first time.
• Or that you can't reliably create SubTextures of SubTextures of textures with a frame / that are rotated, etc. (remember issue #522?)
• Or the complexity of the scale9Grid and tileGrid code.

A subclass wouldn't solve that, because you often don't know you are working with a subclass. You get a Texture instance and work with it — you don't care what's the actual class. You expect it to work everywhere an API accepts that class. And you should!

That's why I'd rather not mix those two concepts. For polygon data, we already have a perfectly suitable class available in Starling 2: Mesh. We're just lacking a file format to load such meshes — and here's an opportunity to add such a format.

[binouze]

Hi, I think about game dev process:

I think I wouldn't start using polygon texture for development process as the texture generation time is mush longer and for testing purpose we sometime need to change texture atlas lot of time. I think I will add it later to optimize render time or textures size later when the project will be more advanced, and maybe i will need to change from one to another for performance testing.

The problem is you make 2 different display class (Image and Mesh), during development it would be very difficult to switch from one to another.

Or maybe add an utility function to get an Image or a Mesh depending on the data used.

[PrimaryFeather]

Thanks for your input, Benjamin!

Yeah, I totally see what you mean. That's why I wrote above that we could let the MeshAtlas class support the original TextureAtlas XML format, as well. That way, if you want to be able to switch between the two formats during development, you could simply use that class from the beginning.

I might even allow it to return Image instances (it would probably support the methods getMesh(name:String):Mesh and getImage(name:String):Image).

Returned images would then be (internal?) subclasses that throw an exception when you want to use them with tileGrid or scale9Grid, though; other than that, they would act just the same as the purely rectangular objects you'd get from the un-polygonized texture.

In any case: you would just have to make sure you use MeshAtlas instead of TextureAtlas in your game. With that decision made, you can create the atlas instances in any way you want during development.

What do you think about that?

[b005t3r]

What @binouze said is a very good point. It has to work transparently.

@PrimaryFeather, so just add polygon data to a Texture and we're good. I think it's the best way to do things. I also understand it's not ideal and things like scale9Grid and tileGrid would not support it, but it's still a much better solution that creating Meshes. What if I want to subclass a Mesh? This approach won't be possible anymore.

But if you really need to have a separate thing for polygons, create something like PolygonData which would contain vertex/triangle data and a corresponding texture as well - it should be fairly easy to work with something like that. But don't force us to have a build-in thing as the only way for creating Meshes (it would be just like creating Images from the TextureAtlas and this would make simple subclassing, which is very handy in some cases, a real nightmare).

[PrimaryFeather]

It has to work transparently.

That's exactly what I'm trying to explain here — it won't work transparently if I simply add the polygon information to the Texture class. It will look like it, but all will implode at runtime.

Let's say Benjamin starts working on his game, and he uses the standard rectangle packing algorithm in TexturePacker during development — because that's faster (makes a lot of sense).

Now, in the final phase of development, he switches to Polygon-packing. Since he's always getting "Textures", that should work, right? Just change that switch and you're done.

Well, except for his Buttons. They suddenly crash on initialization, because he used a scale9Grid on them. Before, the buttons worked fine, because they were standard textures. Now, it's textures with polygons, and that's not supported.

Hmm, and his Bitmap Fonts suddenly do not work any longer, either. He added the BitmapFont texture to his main texture atlas — a good idea, because it allows him to avoid a few draw calls. However, now Starling wants to create a SubTexture that cuts out a letter. That will raise an exception, because you can't create a SubTexture from a Texture with a polygon (and TexturePacker optimized the outline of that font texture).

Damn, the background of one of his messageboxes used a tileGrid. Crash.

And over here, he created a SubTexture from an atlas texture, because he wanted to re-use a small part of it. Crash.

...

You get the idea.

But don't force us to have a build-in thing as the only way for creating Meshes [...]

I don't see how I'm forcing anything upon you. As I said, there already is a perfectly fine implementation of Meshes in Starling 2, called Mesh. All I'm offering here is a format to read meshes from a file, and to get TexturePacker to write just that format.

:confused:

[PrimaryFeather]

BTW, yes, I do understand that subclassing is a problem — you can't use those polygons e.g. for a Button or a MovieClip. That is a problem! But adding the polygon data to the texture class does not work (as described above), and creating a class like PolygonData does not help with that, either.

[b005t3r]

You get the idea.

I do :) But in your example there should be a step in-between the two, or even better an iterative process.

Benjamin switches to a polygonal format. He sees some things don't support it. He switches back to a rectangular format for these. He checks his project again. He fixes some other errors and finally he gets a working, optimized project. Isn't it how we do it all the time? It's not a simple flip of a switch.

But adding the polygon data to the texture class does not work

Yeah, except that it does :) It just won't work for all of the use cases and that's perfectly fine. It's going to work for the others and that's great as well. All that has to be done at some point when creating a project like that is splitting your textures into two atlases - one for the rectangular textures, another for polygonal ones - and maybe merging them together into one "parent" atlas to optimize draw calls. In effect you're going to get rectangular textures and polygonal textures coexisting in the same atlas (this is possible with some fiddling even with v1.8 - I know it works, because this is how I did things in my current project).

I can't really see where you see this huge issue with adding polygonal data to some textures. It's going to be optional, some people will use it, others won't, some features like tileGrid won't support it, but that's fine - you'll just throw an error there - people will have to use rectangular format for these. I know currently TexturePacker does not support mixing both formats in one atlas (it kind of does, smaller bits are still packaged as quads), but maybe it will one day, so you won't have to split your sprites between two atlases. It's all doable and for me sounds more convenient than being forced to never subclass Mesh (and I'm sure I'll have to).

[PrimaryFeather]

Benjamin switches to a polygonal format. He sees some things don't support it. He switches back to a rectangular format for these. He checks his project again. He fixes some other errors and finally he gets a working, optimized project. Isn't it how we do it all the time? It's not a simple flip of a switch.

Well, you said before that you want it to work transparently — and that's the point I was trying to make: that it doesn't.

Sounds to me as if we agreed on that point. :wink:

You brought up one other point that's really important. If TexturePacker decides one day for the user to pick which objects should be stored as polygons, we need one format that supports both use-cases. That's a strong argument against my MeshAtlas idea.

I'll let this whole thing marinate in my head for a few days. Perhaps I can think of an approach that everyone is happy with.

[b005t3r]

By transparently I meant "almost transparently" :) But still, with Mesh atlas it would be even less transparent - you'd have to start using polygonal textures from the start, it would be quite a complex task to switch in the middle of a project.

And about Textures being not only a pure in-memory texture representation, but also having additional metadata attached - I think this is a huge win in Starlings overall architecture. Take scale for example - creating a game with different scale textures in Starling is almost completely transparent, you only need to load the right atlas when the app starts! Also solid textures and sub-textures - these are great, they let you use a set of UVs in textures local coords (0-1) no matter where the texture is located on a bigger one. And that bigger one can be a part of even bigger texture and you won't even know!

Seriously, this is a great design choice, much better than what they did in libGDX, where textures just describe an image data in memory - there's no frames, scaling, parents, etc. And I can already imagine polygon data is going to push this even further. And not much has to be changed, so even people who started working on their stuff when this feature wasn't there will be able to easily switch to using it if they feel like it (which would be rather hard with the MeshAtlas approach).

And, as a general rule, from my experience it's always better to give people a set of building blocks which fit together nicely that a black-box solution that is going to do things for them. The second option always breaks down when it comes to doing things in a more custom manner.

[binouze]

So much argumentations since yesterday ;)

so even people who started working on their stuff when this feature wasn't there will be able to easily switch to using it if they feel like it

I think @b005t3r is right on that point, it would be so good to decide to update an older project (or a current in-dev project) to use polygonal texture, I think that the solution to throw an error when we try to use a scale9 or tilegrid (are there the only limitations ?) on an Image using a polygonal texture is a good thing an not so frustrating.

[PrimaryFeather]

Thanks for the additional feedback, guys!

@b005t3r yes, I also think that the Texture architecture turned out great. That's exactly why I'm so hesitant about all this, because I don't want to ruin that. But I heard you, I'll definitely see what I can do to make this as transparent as possible.

[zzyyxxaabbccoo]

really very need this feature