Allow autotile to rotate/flip tiles when finding a matching bitmask

[SquiggelSquirrel]

**Describe the project you are working on:*** I have been learning to use autotiles for a 2d rpg and an isometric strategy game

Describe the problem or limitation you are having in your project: When creating the texture for a tileset, I find myself spending a lot of time rotating/flipping tiles to create a full set. This results in texture files that are much larger than they need to be, and is especially a hassle when creating animated textures.

Describe the feature / enhancement and how it helps to overcome the problem or limitation: Allow the developer to specify what transformations can be applied to each subtile of an autotile set (e.g can the tile be flipped horizontally, can it be rotated, can it be flipped on either/both axes). This would allow much smaller texture files — e.g. a functional tileset to be created with only four tile textures - one inner corner, one outer corner, one edge, and one filled tile.

Describe how your proposal will work, with code, pseudocode, mockups, and/or diagrams: I have a pull request here: https://github.com/godotengine/godot/pull/39046 Some examples: It may be possible to add a configuration via gui, but I didn't feel confident enough to add that myself.

If this enhancement will not be used often, can it be worked around with a few lines of script?: As far as I can tell, work-arounds would involve either creating a much larger texture file for the tileset, or manually selecting each tile without using the autotile functionality.

Is there a reason why this should be core and not an add-on in the asset library?: I believe this would be used by many, possibly most, tilemaps if it was widely available as standard.

[golddotasksquestions]

Only having to draw 4 instead of 48 tiles would be highly desired and immensely improve the workflow for fast autotile setup.

However this would also mean having the exact same tile at all rotated positions, which would not always be desired. So I would propose if this hopefully gets implemented one day, to make it a 4th option in addition to the already existing 2x2, 3x3 minimal, and 3x3. Maybe call it 2x2 minimal "2x2 flipped&rotated", "3x3minimal flipped&rotated", "3x3 flipped&rotated" or something and also provided example textures for each: https://github.com/godotengine/godot-docs/issues/3316

[SquiggelSquirrel]

@golddotasksquestions This PR allows the developer to specify which (if any) transformations are allowed, so a tile could be specified as only allowing flip_h and no other transforms, or only rotate 180 and nothing else.

Specifically, the allowed transforms use the following flags: FLIP_X : 1 FLIP_Y : 2 FLIP_BOTH: 4 (equivalent of rotate 180) TRANSPOSE : 8 TRANSPOSE_FLIP_X : 16 (equivalent of rotate 90) TRANSPOSE_FLIP_Y : 32 (equivalent of rotate 270) TRANSPOSE_FLIP_BOTH : 64

So to specify "any rotation", one could use (4 + 16 + 32) = 52

The default value is 0, meaning no transforms allowed.

For isometric tilemaps, the flags specify which transforms can be applied to the bitmask, and an appropriately mapped transform is applied to the texture - e.g. a transposed bitmask maps to a horizontally flipped texture.

For convenience/readability, I've also added the following constants: FLIP_ANY : 7 ROTATE : 52 ANY : 127 ISOMETRIC_FLIP_X : 8 ISOMETRIC_FLIP_Y : 64 ISOMETRIC_FLIP_BOTH : 4 ISOMETRIC_FLIP_ANY : 76

In the event that more than one subtile could fit (based on allowed transformations), a random subtile is selected, using the existing priority system. In the event that a single subtile could fit with more than one transform, a random transform is selected (but the subtile itself is still only counted once for the purposes of priorities), with equal weighting on each transform.

This feature should function for 2x2, 3x3 minimal, and 3x3 tilesets.

Allowed transforms are specified on a per-subtile basis, so different subtiles within the same autotile can have different allowed transform masks.

[SquiggelSquirrel]

For example, here's a quick 3x3 with only FLIP_X enabled:

[golddotasksquestions]

The already existing Tileset UX and UI is terrible and overly complicated. Imho it has to be massivly simplified (while making it more intuitive and keeping all existing functionality) not complicated even more.

I'm not sure what you are trying to show with your last comment. The above FILP x 3x3 example screenshot clearly does not work as a Tileset.

There are 16 unique tiles in a 2x2 set, 48 unique tiles in a 3x3 minimal set (no diagonals), and 256 unique tiles in a complete 3x3 set. If Godot would be able to automatically flip and rotate tiles it would greatly reduce those numbers, but at the same time it would limit what could be done with such a tileset. Visually, tileset textures used in that mode would have to always be flat and unlit.

I am worried that adding such a feature make the already incredibly hard to learn and super confusing Tileset UX even more repulsive. That's why I proposed to pack this in a separate mode. This would make it much easier to explain. Like "use this mode only for X" instead of adding checkboxes.

[SquiggelSquirrel]

This pr does not change the UI in any way. Devs can continue with the existing UX, unaware that this feature even exists - similar to _is_tile_bound, just because the feature is there doesn't mean the average entry-level developer needs to be made aware of it. This could simply be a powerful tool for any developer who feels confident enough to read through the APIs and code up their own extensions.

As I see it, any future work to add different "modes" to the UI/UX would need the underlying objects to support this kind of functionality - this pull request focuses on what the TileSet and TileMap objects are capable of via API calls, not on exposing that functionality via the UI.

That said, I feel like reducing this to an all-or-nothing mode greatly reduces its usefulness. Consider:

2d platformers may want horizontal flipping, but not vertical flipping, since these games often have "floors" and "ceilings" that look completely different.

Brick walls and wooden floorboards (to name a couple of examples) can often be flipped in either/both axes, but should not be rotated.

Rotation without mirroring us less useful, but makes sense for things like animated conveyor belts, hazard stripes, sci-fi maps with text & icons on the floor.

Allowing all transformations allows the maximum amount of variety from the smallest texture, which is useful for "natural" looking terrain (especially for RTS games).

Isometric tilemaps require different handling to square ones, this is not currently a property of the tileset so handling it automatically would require additional complexity. Isometric "floor" tile could generally be rotated on any axis, while "wall" tiles would only be flipped horizontally. The aforementioned limit on sci-fi maps would apply, but for isometric that means one might want to only enable 180-degree-rotation, and nothing else.

I can see that allowing transformations to be specified on a per-subtile basis instead of a single value across the entire tileset might be overkill, but again, any GUI/UX layered on top of this could easily provide more limited functionality, while leaving full granular control to the more advanced developer.

Having updated the above example, I hope the following demonstrates how this could be useful for a 2d platformer. I realise it's not the most complete or polished tileset ever, but it does demo how horizontal flip only and 3x3 tilemaps can be useful to a 2d platformer.

[golddotasksquestions]

I can't speak about the underlying C++ code or API you want to add, I can only speak from a user perspective. And the current Tilemap user experience is already abysmal.

I see this approach

This pr does not change the UI in any way. Devs can continue with the existing UX, unaware that this feature even exists - similar to _is_tile_bound, just because the feature is there doesn't mean the average entry-level developer needs to be made aware of it. This could simply be a powerful tool for any developer who feels confident enough to read through the APIs and code up their own extensions.

as the root of the problem.

Some contributors seem to like to add functionality, but not tools for others to easily use that functionality and incorporate it to everyones workflow. If you only created that feature for yourself, and build your own fork, then that's totally fine. You know how to use it because you coded it. If it gets added to the Godot core, others will want to use it too, it has to get documented, explained, maintained, and integrated. If you build an Ui UX around it that is intuitive and self-explanatory, there is much less explaining, and documenting to do. Your feature will actually get used by the community and won't just bloat the engine.

Tileset Wildcards for example, (probably) solve a number of issues people regularly have on the community channels, but they are not documented at all. I say "probably" because how would I know how to use them? I don't. The UI does not teach me. There is no tutorial in the docs, not even API documentation. I have to resort to wiring issues and then maybe, if I'm lucky a contributor will explain that specific usecase.

I would really really love to have and use this feature, but please, let's not pile more functionality on the already overloaded complex Tilesets without a proper integration.

[SquiggelSquirrel]

Fair enough.

Then, from a UX perspective I'd say:

1. This functionality should be kept separate from the Autotile Bitmask Mode. The question of whether we're dealing with a 2x2 bitmask, and 3x3 bitmask, or a 3x3 minimal bitmask (or any other options that get added later) is entirely independent of what transformations are allowed. Mixing them up is only going to add more complexity in the long run.
2. I still think an all-or nothing approach would be too limiting, but I suspect most use-cases could be covered by answering three questions: a. is horizontal flip allowed? b. is vertical flip allowed? c. is rotation allowed?
3. From a technical standpoint, if the above three are supported in the UI, there's arguably no reason to not support all other variants in the API, for those with a bit more coding know-how. In fact I'd say restricting it to one of the 8 possible combinations produced would only add unneeded complexity to the code.
4. The above three could perhaps be condensed into a single drop-down of six allowed transform settings: "NONE", "FLIP_H_ONLY", "FLIP_V_ONLY", "FLIP", "ROTATE", "ANY".
5. This could be set for the entire autotile, along with name, bitmask mode, etc. if that makes anything easier. I'm not sure I see any benefit to adding this restriction, though. Mainly I'd prefer to see improvements on the UX for applying settings to a single subtile, or bulk-update of many subtiles at once.
6. We need more/better documentation, agreed. I'd be happy to write some documentation for how this feature works and could be used at a scripting level, I assume most people using Godot are willing to write at least a little bit of code. I held off before now because I was hoping to know if any changes were needed to the technical side of things first. I kinda figured getting a technical implementation that worked and was approved by the devs would be the first step, documenting it would be the second, adding a decent UI would be third, and documenting that UI would be fourth.
7. I also figured that improved documentation for the existing UI and API was a separate issue, one that had already been raised and was perhaps already being worked on - maybe by one of the contributors who had already worked on the tileset functionality before, since I've not been involved before now. That said, developing the pull request for this involved a lot of me reading through and making sense of the existing code, so I now have a better understanding of how the undocumented features work, and I may have a go at writing some documentation or creating a tutorial when I have time.

[golddotasksquestions]

I kinda figured getting a technical implementation that worked and was approved by the devs would be the first step, documenting it would be the second, adding a decent UI would be third, and documenting that UI would be fourth.

When I'm designing a feature, I usually approach it from the exact opposite direction. I try to imagine myself in the users position, try to picture what a user would want to do exactly and how a user would want to do it. The targeted end goal experience informs how a feature is implemented, what the feature needs to be able to do, what not. Like: "I want the user to see this, then the user only has to click this, and the result is that". If I can't test my feature against that targeted flow, how would I know if it is successful and does what it is supposed to? If the technical implementation fundamentally works without testing it against the flow you risking of missing the features point. There is also the risk of life coming in your way and the UI/UX never being implemented. Badly documented and baldy implemented features literally bloat the engine with things that are hardly used and are at best are extremely elitist imho (only of use for the few who can study and understand the source code)

[DleanJeans]

Hope the PR gets merged some day.

I was participating in the Miz Jam and was given an asset pack with these tiles:

With the current state of Autotile it's impossible to use it out of the box. So I had to open an editor to duplicate and rotate and flip around the tiles to end up with a tileset with almost 4x the size, which was totally unnecessary:

[boruok]

I have only one question: why PR doesn't merged yet?! this is total game changer!

Did some monkey testing with square and isometric projection, works flawlessly, sometimes tiles under tiles mouse keep changing if you keep pressing, but that was my fault with incorrect bitmask layout.

here is my files if you want to play with: Test.zip

[Abtinestor]

how about taging tiles that made to be able to rotate or mirrored in the tile edit tab ? edit: sorry if it was already said or thought of

[KanjiCoder]

I am working on an engine that does something like this. I call it "Auto Latching" , "Auto Mirroring" and "Auto Rotating". https://imgur.com/gallery/g0FXC6L

I also have two 16-subtile auto tile formats. One I call a "fourset" https://imgur.com/a/zHKteUO

The other I call a "golaset" , which is just a standard 16-sub tile auto tile set, but laid out in an artist friendly way.

I am just making up words! But I can do that, it's my engine.

The code is open source. Here is my tutorial series where I document EVERY LINE OF CODE. www.tinyurl.com/BOMB-PLAYLIST-001