[Tiled] Improvements to the Tiled package

[craftworkgames]

Hey guys,

There's been quite a few questions and issues raised recently about improving the Tiled renderer and the API around it. I've been wanting to make some improvements to our Tiled stuff for quite some time now so it seems like the time has come to actually do it.

I've raised this issue to try and gather the requirements into one place so we can discuss how this all fits together. Here's a list of the relevant issues and questions I've found so far (some of these are closed but they have lots of good information in them):

• 469 Tiled objects render order

• How do I go about adding a player object onto a fixed isometric TiledMap object?

• 267 Clean up the Tiled API to remove any inconsistencies

• 230 Combine TiledMap-Rendering with Entity-Rendering

• It's also often good to look at how other libraries do it.

The way I'd like to approach this work is to start by creating some demos that show how to use Tiled Maps in a few different scenarios. The demos don't need to be full games, but they should be realistic examples of how to use Tiled maps in the context of a game.

In other words, we need some demos that render some different types of maps (orthogonal, isometric) and also render sprites between the layers and tiles respectively. This is a super important distinction, because in the past our demos have only really focused on getting the maps to render technically but when you're making a game that's really not the most important thing.

Our demos are our tests and they need to be treated that way. If we don't have a unit test or a demo for a feature then we run the risk of breaking it by accident and that's not cool.

Lastly, our demos are often the first impression of the library and people often use them to learn how to get started. Keeping the demos up-to-date when new pull requests come in helps to confidently merge the changes and see what those changes are actually doing.

[fluxtheory]

I'm still a novice dev and new to extended at that, but that libgdx example would be how I would try and implement it. Give us the ability to extract individual layers from TiledMaps, and from those layers the ability to get information from individual tiles. Add on top of that a dynamic container to store all this information in that we can use to modify the base map and to track game state changes, and that's sufficient enough for most cases.

[kimimaru4000]

I've started getting into Tiled, and I generally prefer to render things my own way rather than use the built-in draw methods. That code is just a test on an Orthographic map, but getting it to work was a little harder than I thought. That said, I have a few issues with the current API:

1. TiledMapTileset.GetTileRegion accepts a local identifier, but TiledMapTile has a global identifier. Passing the global identifier in as the local parameter looks to work, but I'm not sure if this will work for all cases.
2. TiledMap.Layers seems redundant to me. We already have ObjectLayers, TileLayers, and ImageLayers; is grouping them all in one property necessary?

Lastly, I have only one tileset, but if I had another, I wouldn't know how to find out which tileset is used for a particular tile. I can look into this further later to make sure the API supports it.

As fluxtheory mentioned, I would love to be able to change the base map at runtime. I would also like to see a method that loads a Tiled map without having to use the content pipeline.

Please correct me if I'm wrong about any of the above; as previously mentioned, I'm still fairly new to Tiled.

[stefanrbk]

TiledMapTileset.GetTileRegion accepts a local identifier, but TiledMapTile has a global identifier. Passing the global identifier in as the local parameter looks to work, but I'm not sure if this will work for all cases.

This I will definitely look at this. I didn't catch it when I was last working on the Tiled library. In the TiledMap, tilesets should be the structures with local identifiers and the tiles in the TiledMapTileLayers should use global identifiers.

Lastly, I have only one tileset, but if I had another, I wouldn't know how to find out which tileset is used for a particular tile. I can look into this further later to make sure the API supports it.

So, the answer here is all in the global identifiers. Each loaded TiledMapTileset has a first global ID value which defines what the ID of the first tile is for the map. If the global identifier you are checking is greater than or equal to that first global ID and is less than the first global ID plus the number of tiles in that tileset will tell you if the tile is in a particular tileset. I think there is a method to find this out already, but I can't remember.

As fluxtheory mentioned, I would love to be able to change the base map at runtime. I would also like to see a method that loads a Tiled map without having to use the content pipeline.

This is something else I want to look into as well. It would be nice to be able to change the map at runtime, but I'm still wrapping my head around how we render the models on the GPU so we can insert SpriteBatch between the layers as well as adding real TiledMapObjectLayer support as we currently can't render objects.

TiledMap.Layers seems redundant to me. We already have ObjectLayers, TileLayers, and ImageLayers; is grouping them all in one property necessary?

It is redundant, but I believe that is intentionally so. When the TiledMapRenderer draws, it only references TiledMap.Layers, but I feel this functionality will be helpful when we finish adding mutability into the Tiled library. Makes it easier than having to type check each TiledMapLayer in the list and manually propagate TiledMapGroupLayers. I.E., all TiledMapTileLayers are in the TiledMap.TileLayers list, no matter if it is in some TiledMapGroupLayers or not.

I hope that helps? I'm currently researching the SpriteBatch thing with render order so we can make rendering easier, so I can look more into these then!

[craftworkgames]

Give us the ability to extract individual layers from TiledMaps, and from those layers the ability to get information from individual tiles.

I believe it's possible to already do these things with the current API but it's not completely obvious how to do it. We'll certainly be looking for ways to improve this through a better API and better documentation.

Add on top of that a dynamic container to store all this information in that we can use to modify the base map and to track game state changes, and that's sufficient enough for most cases.

Yes, that's pretty much the idea. Although, exactly how that's going to work still needs to be fleshed out. There's quite a bit of complexity making it both easy to use and maintaining a reasonable level of performance.

TiledMapTileset.GetTileRegion accepts a local identifier, but TiledMapTile has a global identifier. Passing the global identifier in as the local parameter looks to work, but I'm not sure if this will work for all cases.

I agree this is quite a confusing part of the API. When I wrote the Tiled API originally it followed pretty closely with the TMX map format which is where the confusing local vs global identifier stuff comes from.

Things have probably changed over time in both our API with various pull requests and there may have also been changes in the TMX map format in newer versions of the editor. I'll have a think about how we can improve this stuff, maybe with some helper methods that handle some of the more complicated bits.

TiledMap.Layers seems redundant to me. We already have ObjectLayers, TileLayers, and ImageLayers; is grouping them all in one property necessary?

Actually, looking over this code right now it seems it doesn't work the way I originally intended. Somewhere along the line the purpose of these properties seems to have changed.

The original intent was to only store the layers once in the Layers property and provide the ObjectLayers, TileLayers and ImageLayers properties simply as a convenient way to filter to only those layers.

I'll take a closer look at this code to see if I can figure out why it was changed this way and see if it still makes sense to do whatever it's doing.

I would also like to see a method that loads a Tiled map without having to use the content pipeline.

Yes, absolutely. One of the things I'd like to do in 2.0 is have the ability to load any of our content without needing to use the Content Pipeline. All of the content we support is stored as XML or JSON and now that we're using .NET Standard I think it's easy enough to move the loading code into the run-time libraries. That said, we've still gotta work out the details of how to actually do it.

I'm currently researching the SpriteBatch thing with render order so we can make rendering easier, so I can look more into these then!

@stefanrbk Thanks for your help here. It seems like you've got a pretty good understanding of how things work in the Tiled library. I'd like to start working on some of this stuff but if we're both making big sweeping changes in the same code it could result in some painful merge conflicts.

For now I'll start doing some ground work that shouldn't mess with what you're currently working on. We should probably have a chat about it soon though so we can get on the same page.

[stefanrbk]

The original intent was to only store the layers once in the Layers property and provide the ObjectLayers , TileLayers and ImageLayers properties simply as a convenient way to filter to only those layers.

When I was adding group layer support, this is what I was aiming for with those properties. Layers is what the renderer iterates through when a map is passed in to render and stores all the layers as they would appear in the Tiled editor. The filtered properties should still be working as you said. There is a bit of complexity that had to be added so layers within groups don't get added twice into Layers, but it should still work that way. I noticed what the filtered properties were when I was testing the group functionality. I somehow noticed the layer count significantly changed when I loaded the map I edited to include a group layer. It rendered alright, but it was not counting any layers inside groups. I'll have a look later and see if I didn't get that right or if it was something before the overhaul.

I'd like to start working on some of this stuff but if we're both making big sweeping changes in the same code it could result in some painful merge conflicts.

My concern exactly. That's why with the last overhaul I did the WIP, so what I was doing could be visible.

We should probably have a chat about it soon though so we can get on the same page.

That's fine with me! With being on opposite sides of the globe, might be easiest on a weekend. :smile:

[lithiumtoast]

@craftworkgames

Add on top of that a dynamic container to store all this information in that we can use to modify the base map and to track game state changes, and that's sufficient enough for most cases.

Yes, that's pretty much the idea. Although, exactly how that's going to work still needs to be fleshed out. There's quite a bit of complexity making it both easy to use and maintaining a reasonable level of performance.

I talked about this before. References here, here, my proposed solution here.

I honestly think that people using SpriteBatch for everything is one of the reasons why XNA and MonoGame are so popular for 2D game development and not so much for 3D game development. It seems most people who want to make games using MonoGame or make games using any framework or library with just don't understand how rendering works. These people then create solutions which are by definition technical debt.

[stefanrbk]

Thank you for linking those references. That is why before I try to put any time into creating any sort of solution I want to study how the rendering works as fully as possible. I work for a cell phone carrier, and we are trained heavily not to just slap a bandaid over a problem, go to the source (pun somewhat intended)

I want to understand how everything is rendered so any changes will to improve the library.

[stefanrbk]

@tdeeb

TiledMapTileset.GetTileRegion accepts a local identifier, but TiledMapTile has a global identifier. Passing the global identifier in as the local parameter looks to work, but I'm not sure if this will work for all cases.

This I will definitely look at this. I didn't catch it when I was last working on the Tiled library. In the TiledMap, tilesets should be the structures with local identifiers and the tiles in the TiledMapTileLayers should use global identifiers.

I just researched this and TiledMapTileset.GetTileRegion is local to the referenced tileset. This is just a helper to grab the Rectangle where the specified tile is in this tileset. TiledMapTile does use a global ID, and this is technically a different ID. You would use TiledMap.GetTilesetByTileGlobalId to find the tileset, then TiledMap.GetTilesetFirstGlobalIdentifier to find the global ID of the first tile of that tileset as it is in this map, and finally subtract the global ID from the first ID of the tileset to find the specific tile. Typing this out does show me the complexity here... From what I can see, this isn't something which is even too helpful to know at runtime with our current API. It might be more helpful when we have mutable TiledMaps. But I could be wrong. If anyone has a need for this I can't think of, I can clean this up.

[lithiumtoast]

@stefanrbk https://simonschreibt.de/gat/renderhell/

[stefanrbk]

Ooooo! Thank you!

[craftworkgames]

I've been making some progress on a new Tiled Maps demo that should cover a bunch of the issues mentioned here.

I ran into #295 and I'm going to try come up with a better solution to that next.

[lithiumtoast]

@craftworkgames

Just to be clear, if you are trying to avoid the content pipeline, it's fine to that all that black magic of texture packing is out or in the content pipeline. The problem in general is called bin packing. It's an NP-hard problem. My crude proposed solution to the problem goes like this. Start with a maximum sized texture for the platform. If I remember correctly with the XNA Reach this was 2048x2048? Then create a priority queue of rectangles sorted by the rectangle area (width * height), largest to smallest. Take a all the textures that need to be packed and sort them by rectangular area, largest to smallest. To begin insert the whole region of the texture into the priority queue. Then take the next largest texture to pack and the next largest rectangular area that is free from the priority queue. Place the texture by copying the pixels from source (tileset texture) to destination (packed texture, texture atlas). Record the information of the region into the texture atlas using a dictionary so it can be looked up later in constant time. Split the free region into appropriate rectangles and place them in the priority queue. There is possibly the rectangle to the right and below, which way to split should maximize for largest area. Recursively do these steps until every texture is placed.

For tilesets, usually the tile size is uniform across maps, so everything gets packed nicely into one super tileset, if they can all fit. If that is the case then it's even possible to re-write the map in memory to use the one super tileset for all tiles instead of multiple tilesets. The user shouldn't really care about this detail as long as they can use their tilesets using the texture atlas. With a texture size of 2048x2048 and tile size of 16x16, thats room for 16384 tiles (128 * 128). That's a lot of tiles. I think 4096x4096 texture size might be doable for most platforms too. In that case there is room for 65536 tiles. I'm pretty sure in most cases all the tiles will be able to fit into one texture.