YoYoGames / GameMaker-Bugs

Public tracking for GameMaker bugs
22 stars 8 forks source link

Instant depth sorting with performance control #3009

Open ghost opened 1 year ago

ghost commented 1 year ago

Is your feature request related to a problem?

Many beginners who try to make top down games run into the issue of depth sorting instances. Often we see the depth = -y, the grid sorting method, even using the depth buffer.

All of these comes with positives and negatives, but I'd like a solution that gives us instant control over instance priority on a layer, just like we already have in the room editor (the order in which we place instances in the room editor).

After toying around, I found that Game Maker already seems to handle instance priority at runtime in a somewhat usable way to achieve this.

More can be found in this forum post: https://forum.gamemaker.io/index.php?threads/depth-sorting-and-internal-game-maker-instance-priority-an-insight.105205/

My idea was that I simply -directly- reprioritize instance order by user the "layer" variable (after a grid sort). The benefit is that no event_perform(draw event) "draw event hijacking" needs to be performed every step, which is quite costly for performance.

With this, I can now depth sort instances every few steps, thus spreading out the performance hit a lot, making it more performant (as well as easy to understand).

I feel like this should be possible in a single layer function, something like layer_instances_sort_depth(_time);

the _time argument being how many steps it takes between a depth sorting cycle, to give user control when it comes to performance.

Right now, this relies on how Game Maker internally handles instance priority at runtime, and my system (blindly) manipulates that in such a way that I end up with high performance depth sorting, and still full access to instance draw events, without any additional setup.

Fast, easy, beginner friendly.

But, it's the user control that is missing here. As it stands,this is still somewhat black box behavior, and my request is that we get more control with this principle in mind.

Describe the solution you'd like

A function that gives instances a depth priority number, related to the exact pixel it is on.

This function then sorts these instances according to that number, possibly with the option to sort top to bottom as well as left to right.

Then, by optional frame count, let the function reprioritize the existence of instances on a layer, and create a depth sorted effect for them.

All code and examples can be found in this forum post on how I got it to work:

https://forum.gamemaker.io/index.php?threads/depth-sorting-and-internal-game-maker-instance-priority-an-insight.105205/

As you can see, i ran into an issue when I was applying object culling, something I solved. I believe this happened because I simply didn't have "true" access of instance order at runtime, and I found a way around that.

It seems to me extremely simple to implement,

and it would make development a lot easier for many, especially beginners, as depth sorting often relies on either far more advanced techniques, or techniques that are simply really bad for overall performance.

Describe alternatives you've considered

Alternatives I considered:

Depth sorting every frame through a grid and using event_perform(draw event) => Too costly

Depth = -y => No control over "left to right" depth sorting

Using shaders and the depth buffer => Trouble with pixel transparency + very advanced, especially for beginners

Priority queues => Too costly

Additional context

The reason I needed a fast and easy solution, is because I am making a game that can have thousands of depth sorted instances within the view (crops on a field, ores in mines,....)

I also wanted something that makes it easier for beginners

Totobal5 commented 1 year ago

oh yeah, a native y-sort function would be great

ghost commented 1 year ago

oh yeah, a native y-sort function would be great

Not just y sort, but an x and y sort :) The example in the link in the forum post shows exactly how to do it, but we'd need user control on instance order at runtime to be 100% certain that what I did there is safe to use across newer Game Maker versions.

There would be limitations, kind of like how now there is a limit on the depth variable outside the range of -16000 and 16000.

In my systems, room sizes up to 9999x9999 are allowed, but it can be made even larger (although why such large rooms right?)

The big benefit here is the quick implementation of depth sorting with a very fast cpu based solution.

AtlaStar commented 1 year ago

isn't this just layer_add_instance or layer_element_move but while also trying to assume what the end user wants? With proper layering, the blackbox behavior won't matter because you are utilizing layers in a way that draw ordering in a given layer will not cause issues, and more often than not you don't need to re-sort the depth of things that are static. So realistically only certain managed entities/props need to ever possibly swap to a new layer at all to ensure proper depth, which mostly falls to the moving entities in a game.

Basically, the proposed function would need to assume far too much to be useful, when most things can just be put on a proper layer to begin with and never have to change. Things which do change can be plopped onto in-between layers that sit between your "multi-tile" prop. All that is needed is to place those sprites/tiles on a special layer that has sub-layers, and have it at the same depth as your primary layer. Then it is just testing if your sprite intersects with some bounding rect to determine which part of the special layer/sub-layer your instance should be moved to.

An example would be having your trees on this special layer, and normal ground in a sublayer of it. When an entity is in the bounding rect of the tree sprite less some padding, move it to an instance layer that sits between the ground sublayer and the tree layer. You can even then add a layer callback to that special layer that tests the in-between layer for the existence of the player, and if the player is on that layer change the blend mode or set a shader so that the trees become semi-transparent.

In short; if you need convoluted depth sorting, you are likely doing something wrong to begin with that understanding layers better would solve.

ghost commented 1 year ago

isn't this just layer_add_instance or layer_element_move but while also trying to assume what the end user wants? With proper layering, the blackbox behavior won't matter because you are utilizing layers in a way that draw ordering in a given layer will not cause issues, and more often than not you don't need to re-sort the depth of things that are static. So realistically only certain managed entities/props need to ever possibly swap to a new layer at all to ensure proper depth, which mostly falls to the moving entities in a game.

Basically, the proposed function would need to assume far too much to be useful, when most things can just be put on a proper layer to begin with and never have to change. Things which do change can be plopped onto in-between layers that sit between your "multi-tile" prop. All that is needed is to place those sprites/tiles on a special layer that has sub-layers, and have it at the same depth as your primary layer. Then it is just testing if your sprite intersects with some bounding rect to determine which part of the special layer/sub-layer your instance should be moved to.

An example would be having your trees on this special layer, and normal ground in a sublayer of it. When an entity is in the bounding rect of the tree sprite less some padding, move it to an instance layer that sits between the ground sublayer and the tree layer. You can even then add a layer callback to that special layer that tests the in-between layer for the existence of the player, and if the player is on that layer change the blend mode or set a shader so that the trees become semi-transparent.

In short; if you need convoluted depth sorting, you are likely doing something wrong to begin with that understanding layers better would solve.

Right now my depth sorting works like this: One layer with all instances that need to be depth sorted on it (how the user designs the room basically) A grid sort of those (active) instances (automatically, every few steps) Prioritizing the instances after the sort Done

This is basically the Friendly Cosmonaut depth sorting technique, which is extremely beginner friendly, used by most already, but can now be done across multiple steps instead of every step, giving a massive boost to performance.

I believe this to be far easier to understand and implement than using multiple layers for static or moving instances, especially for newcomers.

Plus, how many sublayers are required? And, when dealing with potentially thousands of instances, maybe even many moving ones, how much of a setup would it be?

Here, it's the tried and true "one layer, one sort, fast to implement, easy to understand" way of handling things, and now it can be done much, much more performant. Which to me is the less convoluted way, especially because the setup takes less than a few minutes, and you can just say "okay done, the game now depth sorts fast, now let's worry about game content".

More convoluted, to me, seems to be when you have to start using layers and sublayers in all sorts of clever ways, and than move things around between those layers, having to check individual bounding boxes, keeping track of it all.... as you seem to propose. Remember, we are talking about 2.5 depth here, an absolutely critical element of top down 2D games . And if it's absolutely critical to such games, I would think the engines should provide options to implement such things very, very fast without too much setup. For example, Unity and Unreal start projects with a basic camera already created for you. If you want to code a more complex one, you can, but it's there anyway if you want.

I say make that depth sorting as easy as possible to implement with either a function call or perhaps checking a few boxes in the room editor in the instance layer tab. No further setup required, except of course it would be the responsibility of the end user to think carefully about sprite origins, which is already a must anyway. It would certainly be a much more usable and important option to have than, say, another "fill the screen with colored boxes" filter in the next GM update.

So how about 1 layer, 1 sort, done? Can't get easier than that.

Want a second layer somewhere else be depth sorted sepatately? Just do it there too.

I'd say, especially for newcomers, it would be one of those things that could make Game Maker stand out even more as "THE 2D engine". Gotta make a layer depth sorted? Oh.... Well okay then, that's nice to have!

And if you want to go about it in a more complex way, why not? Use the depth buffer and shaders (which I did, but pixel transparancy became more of an issue + not user friendly I'd say) Use the priority queue method (not performant though) Use the layers and sublayer methods (welcome to managing many layers, doesnt look as nice of a workflow, which frustrated me) Go back to the old grid sort but now every step and redrawing everything (very common, easy, but not performant) Go back to the very old depth = -y system (not a good idea now that layers are the main structure of where instances are)

I tried all of these but kept experimenting, and ended up with what I propose.

The reason is simple: One layer, boom! sorted! Basically, what you ALREADY get in the room editor when placing instances on a layer. There, in the room editor, the instance creation (placement) order is ALREADY respected. My system plays around with that placement order, something the engine already does in the room editor to begin with! This proposal simply expands the use of instance order from the room editor to "at runtime".

So, what you already do when dragging instances on a layer with your mouse in the room editor in order, but now being able to control that order during runtime as well.

In a way, this goes further than simple depth sorting: I want to have that instance priority order be under control at runtime, and depth sorting just happens to be a nice (and the most usable) effect that comes out of it. It's what my system actually does, however I'd think it would be more user friendly and more secure if we just got a function for it and have it be part of documented engine behavior.

Basically, I'm asking for a function that simply does it for us. Noone needs to use it if they don't want to.

Kind of like the new move_and_collide function. You could program your own advanced collision system, or beginners can simply implement move_and_collide now quickly, and then later maybe learn more and expand on it. They didn't really need to give us move_and_collide, but it's certainly a nice function to have especially for beginners.

Depth sorting is a crucial thing to learn for top down 2D games, and beginners might find it a little hard to wrap their head around, especially when it comes to performance. But with my proposal, it would be an easy thing to implement AND remain quite performant, even giving some control over how "accurate" it should be (performance vs time it takes to sort).

1) Drop your depth sorted instances on the layer you want depth sorted 2) Call the proposed layer depth sorting function in a step event (or maybe even check a few boxes in the room editor). And that's all the setup you'd need, basically.

AtlaStar commented 1 year ago

As I said already; you want a magic function that will have to make a lot of assumptions about what the end user wants, because how does the engine dictate how to depth sort beyond what it currently does? Currently anytime you change the depth of something, it literally gets moved to an implicit layer at that depth that is engine managed. So step one is for YYG to completely change that mechanism to some other magic system that just knows what the end user wants....that isn't possible.

So what you actually end up with is needing to run different logic for the depth variable dependent on whether it is in this magic layer type or not, and have it sort the draw layer for you...this is no different than just doing depth = -y except instead of multiple layers, you get one...it also is asinine since it requires you to re-sort the entire collection of elements on that layer.

Basically, there is no way to make this work without having people just go back to using the old tricks, but with some special 'magic' that reduces the amount of managed layers that are created. So a no code option that you first alluded to isn't even possible, and all this does is bring back a paradigm that was supposed to die with gms 1.4 in order to even leverage such a feature.

An actual no code solution would involve special layers and sublayers of them, with Gamemaker handling the switching of instances between these special layers for you, implicitly. You'd be able to do this all with the functionality that exists in the room editor itself, which might take a bit more thought when designing things and an improved UI for that special case, but otherwise would be more intuitive since it is easier to work with things that are providing instance visual feedback.

If you need more precise control, then you should really be using layers and swapping things between them anyway...and as to your question about how many layer's you'd need using the case I provided, it depends. If you had 2 7 tall buildings with the top of one building overlapping the ground floor of the other, 13 would be the upper bound on what is needed, with a lower bound of 2. Basically it depends on the amount of things overlapping each other in a given axis and if that is the depth sorting order you wish to use.

Finally, I want to reiterate that your desire wouldn't allow you to do things like set a layer callback for trees/buildings/etc your player object is behind, since they exist on the same layer, so you'd need multiples of these special layers if you wanted to utilize any sort of layer based effect, blend mode, shader, etc, basically landing you back to what the alleged problem was before as you'd need a way to "swap" into those layers. So what I think we really need is for that mechanism to exist implicitly and for Gamemaker to add layer types that manage the depth of instances if they are intersecting things on that layer.

ghost commented 1 year ago

As I said already; you want a magic function that will have to make a lot of assumptions about what the end user wants

Like move_and_collide?

So what you actually end up with is needing to run different logic for the depth variable

Uh, no? The depth variable isn't used here, at all. It's all the depth of the layer the instances are on. This simply grabs control of the instance order on a layer, which ironically can easily cause a depth sorted effect

Basically, there is no way to make this work without having people just go back to using the old tricks

It works perfectly for me, and I have a massive boost in performance, so that I don't end up going back to the other systems. Maybe i could still go for the depth buffer and shaders, but as I said that's more complicated. Plus, I use the very commonly used Friendly Cosmonaut grid sort technique, a system already used by many. I just optimized it in order to use instance order on a layer, rather than redrawing everything every step.

An actual no code solution would involve special layers and sublayers of them,

My system proves it can be done on a single layer, all I'm asking is to pack the system into a new OPTIONAL to use function. Which should only serve to help newcomers quickly set up something. It doesn't affect anyone already using other systems.

So what I think we really need is for that mechanism to exist implicitly and for Gamemaker to add layer types that manage the depth of instances if they are intersecting things on that layer.

That would also be nice to have

Anyway, i'm simply asking for one additional function, for one specific thing, but it would be incredibly useful for many things. Quickly setup a depth sorted layer? Done => Good for prototyping, good for starting out,

Or, maybe not a specific function for this specific requirement, but rather asking them to just give us control of instance order on a layer already. At runtime, not just in the room editor. After that, we can do with it whatever we need it for, not just depth sorting (which just happens to be a quick, fast and easy results that come out of it).

tabularelf commented 1 year ago
  1. YoYoGames/GameMaker-Bugs#3153 is similar to this (and was rejected within reason)
  2. I’ve seen a lot of depth sorted systems, and unless you absolutely need one layer to rule them all, it doesn’t hurt to actually do depth = -y that much. There was a big fear factor over the new layer system when it was first introduced, but all it does is spawn new auto managed layers and renders things for you. And deletes them when there’s no more instances on top. It’s surprisingly not as bad as looping manually. The alternative would be to have an array full of layers created, and just depth sort them via the array. layer = layers_array[y div TILE_SIZE]. Which would work between 0 to size of room.
  3. Beta has gpu_set_depth and `gpu_get_depth, which with your current rendering system (and zwrite/ztest enabled), could render them in order without any extra complex work. (And it doesn’t break the batch either)
ghost commented 1 year ago
  1. Sorting draw order of instances in a layer #1124 is similar to this (and was rejected within reason)
  2. I’ve seen a lot of depth sorted systems, and unless you absolutely need one layer to rule them all, it doesn’t hurt to actually do depth = -y that much. There was a big fear factor over the new layer system when it was first introduced, but all it does is spawn new auto managed layers and renders things for you. And deletes them when there’s no more instances on top.
  3. Beta has gpu_set_depth and `gpu_get_depth, which with your current rendering system (and zwrite/ztest enabled), could render them in order without any extra complex work. (And it doesn’t break the batch either)

1) It's strange though I think. As there already is an order within a specific layer going on anyway. Just like the instance creation order in the room editor. It's kind of already there, and my system shows it can be manipulated (repriozitizing instances on a layer). It wouldn't even be that much work, just use what's already there but now with user control at runtime, no longer just in the room editor. A single, optional to use, function could do this. Drawing everything in order every frame after sorting is demanding for performance, but with my system here there's a lot more fps real estate with this kind of instance order sorting. I'm simply playing with what the engine already does at runtime anyway.

As for 2) depth=-y is definitely the quickest setup, but I use a lot of things that also overlap left to right, not just top to bottom. In order to make it look good, I really needed a depth sort from left to right, then top to bottom. depth=-y only sorts top to bottom. Plus, you're stuck with the -16000 to 16000 depth range. Imagine many crops like wheats next to each other on a 16x16 grid, but they overlap horizontally. I can now sort instances left to right, top to bottom, even if there is an instance for every pixel in the room, in large rooms (say 9999x9999 pixels). It's basically pixel perfect depth sorting without "falling outside of a depth range".

3) The GPU solutions I've found are indeed, by faaaaar, the fastest for performance. But I'm asking for a beginner friendly optional function, like move_and_collide. (But in reality, I'm really asking for instance order control at runtime, depth sorting is just an effect that comes out of it, but probably not the only use).

ghost commented 1 year ago

Just to compactify my request even more:

It's about getting control at runtime of instance order on a layer. We already have control over how layers are ordered, so why not go one step deeper and also give us control over instance order on a specific layer? Seems like such an obvious optional thing we should be able to do.

We already have that control in the room editor, so why not at runtime?

My system here is simply one aspect that could come out of it, but I'm sure having that extra control over things would also see other uses as well, if you so desire.

That's really the heart of this request, not just my specific use of depth sorting instances.

DiasFranciscoA commented 1 year ago

It cannot be as simple as layer_instances_sort_depth(_time); that system would need to assume you want to sort by y which you might not. So you would need to use a layer_instances_sort_depth(_time, _sort_method); (or something similar to allow for better control) and even that wouldn’t be so simple… since this raises other questions: “is running a GML function on each instance element of a layer faster than other existing approaches?” 🤔 I think it might be complicated to make something general enough while still being (performant). I’m bringing up performance since that seems to be one key requests of the OP.

NOTE: I’m not trying to destroy the idea, I’m just playing devil’s advocate here.. so all these little issues are taken into account, prior to possible development.

ghost commented 1 year ago

It cannot be as simple as layer_instances_sort_depth(_time); that system would need to assume you want to sort by y which you might not. So you would need to use a layer_instances_sort_depth(_time, _sort_method); (or something similar to allow for better control) and even that wouldn’t be so simple… since this raises other questions: “is running a GML function on each instance element of a layer faster than other existing approaches?” 🤔 I think it might be complicated to make something general enough while still being (performant). I’m bringing up performance since that seems to be one key requests of the OP.

NOTE: I’m not trying to destroy the idea, I’m just playing devil’s advocate here.. so all these little issues are taken into account, prior to possible development.

yes, 2 arguments would be needed, that is, if a depth sorting function is all we want out of this: sorting method and step update for the sort. Although that last one isnt really needed, come to think of it.

I think it would be more usefull to simply have the array holding the instances be sorted, through a sort method of your own choosing (like many functions already have).

The downside I think, if depth sorting is kept in mind, is that the instance coordinates matter, so at least one sortby would be one of the methods that can be used as an argument, perhaps sort_by_y_down (which will look the same as depth = -y) sort_by_y_up (which will look the same as depth = -y but upside down)

sort_by_x_left (same but tilted 90°, left to right) sort_by_x_right (same but right to left)

sort_by_xy_left (both, left to right, top to bottom) sort_by_xy_right (both, right to left, top to bottom)

and maybe 2 more to invert those last two vertically.

Come to think of it, the time update wouldnt be needed, that's something the user can do with their own timer code. So in that case, all we end up with in this request, is the request to have control over the order of instances on a layer. The sort methods could be packed in the argument, and each method would do things differently, for example:

by object name, A-Z by object name Z-A by depth sort xy (each instance will have to have a value 10000y-10000x) assigned though

typing that last one makes me realize the performance problem: Right now my system gives that number to non-moving instances in the create event, and for moving instances in the step event in order to update them. Putting that number in the step event of non-moving instances would be unnecessary, as their position doesnt update. So here is where I can see why it could be unpractical: How to separate the moving from the non-moving ones? Perhaps a secondary function to go along with it: One that needs to be placed in the instances themselves where needed: instance_update_depth_coordinate(); or, a sort_by_instance_variable

then you could just assign any variable yourself to the instances: variable = 10000y-10000x; variable = "ABCDEFG"

and sort by that

Still, even with that extra "complication", it doesn't take away from the fact that control over instance order on instance layers would be nice to have, without having to rely on black box behavior. Although, if we just get the option to sort_by_instance_variable, all of this would become extremely flexible, giving us control to sort anyway we see fit.

I somehow seemed to have found a way around it and gained control over that order, but it can change in the future and then stuff like this would no longer be possible. So having documented control over the order would open op options for us, options of what we'd like to do with order of instances on an instance layer.

So maybe not even a specific depth sorting function, that can still be coded by the user themselves, but maybe this time with the possibility of handling instance order without relying on black box behavior.

Like, they give us control over instance order in the room editor. Why not at runtime then?

That last question is really what this is all about. Let us handle the order any way we see fit, both in room editor (by changing the creation order which we already have), and at runtime (so why not? what exactly is it that makes this too much of a struggle to add, because I don't see a reason why this shouldn't be here, as we already do that in the room editor anyway)

DiasFranciscoA commented 1 year ago

All in all from what you are describing.. it looks like you are suggesting a black box system yourself. Which purpose would be to sort instances in a layer. 🤔

Manually assigning “depth” still seems like the most simple and performant goto.

ghost commented 1 year ago

All in all from what you are describing.. it looks like you are suggesting a black box system yourself. Which purpose would be to sort instances in a layer. 🤔

Manually assigning “depth” still seems like the most simple and performant goto.

Right now it's black box behavior, but there definitely is a list or array kept somewhere to have an order to the instances on a layer.

The depth variable only works between values of -16000 and 16000, and works by placing instances on new layers at those depths.

If I play with order of existence on a layer, by tellin instances in order to have their "layer = same layer", I notice I can directly manipulate the internal order of those instances. This then creates a depth sorted effect that can be updated every few steps instead of every step, thus creating a much more performant version of the grid-sort depthsorting method.

So that made me wonder: what is used internally for instance order? An array? A list of id's? And, wouldn't it be nice that we can simply have control over that order with new features or functions (thus bringin it out of the black box behavior and into the documented functionalities).

I don't think we have access to that list or array right now, at least not directly. But it seems that by placing instances back on their same layer, their order of existence gets bumped up to the top, which allows for sorting. So it's in there somewhere, we just require a workaround to mess around with it.

AtlaStar commented 1 year ago

Right now it's black box behavior, but there definitely is a list or array kept somewhere to have an order to the instances on a layer.

The problem is that the current implementation details don't have to stay the same...at a later date YYG could change the implementation to be backed by some unordered structure like a hash-set, or even an ordered structure like an ordered set to prevent reinsertions, if they were to find it would provide some major benefit to their internals. What you are asking for provides a hard dependency on behavior that they have expressed they do not wish to support; the ability to order things in a layer. Right now the order appears to be the insertion order, but that may not be the case a week from now because there is no behavior guaranteed to do so to us as end users.

The depth variable only works between values of -16000 and 16000, and works by placing instances on new layers at those depths.

Correct, but you can also manage this caveat with a little bit of work; don't move the player, move the world, if it is an infinitely generated world. Even at insane screen resolutions you'll have plenty of layers to do per-pixel and even subpixel layers. If it isn't infinite, use layers better...I get the hesitance to learn something new when you are used to older methods, but proper use of layers can really solve a lot of these depth sorting issues if you learn to think in terms of layering.

If I play with order of existence on a layer, by tellin instances in order to have their "layer = same layer", I notice I can directly manipulate the internal order of those instances. This then creates a depth sorted effect that can be updated every few steps instead of every step, thus creating a much more performant version of the grid-sort depthsorting method.

So that made me wonder: what is used internally for instance order? An array? A list of id's? And, wouldn't it be nice that we can simply have control over that order with new features or functions (thus bringin it out of the black box behavior and into the documented functionalities).

I don't think we have access to that list or array right now, at least not directly. But it seems that by placing instances back on their same layer, their order of existence gets bumped up to the top, which allows for sorting. So it's in there somewhere, we just require a workaround to mess around with it.

The above are all literally something that could change tomorrow, which is why you aren't supposed to rely on undocumented behavior in the first place. Your argument for the function only makes sense if this behavior goes from undocumented to documented first, which based on other context isn't likely because it seems YYG does not want to do for reasons that make sense to them. You keep talking about this internal list or array, which it might be right now, but could change to a much more complicated structure tomorrow. Like for all we know tomorrow they could be using a red-black tree with the insertion predicate being based on element-id (hell, that might be the case now for all we really know) which flattens to a sorted map based on element-id. Or they could decide to have the insertion predicate be instance id's if it were to make some internal thing faster.

ALL that said...I do like the idea of some type of auto-sort, I just don't think it should rely on sorting things in layers, but some tool to make it to where Gamemaker sees your thing is intersecting a special layer type and figures out where to place that instance in that special case for you; something I think YYG might actually be a bit more willing to do since it wouldn't introduce dependencies on behaviors that could change.

ghost commented 1 year ago

ALL that said...I do like the idea of some type of auto-sort, I just don't think it should rely on sorting things in layers, but some tool to make it to where Gamemaker sees your thing is intersecting a special layer type and figures out where to place that instance in that special case for you; something I think YYG might actually be a bit more willing to do since it wouldn't introduce dependencies on behaviors that could change.

How about a new instance layer type then, one were we do get full control of instance order? An exception layer with that kind of user control, should they decide the internal mechanism of instance order is to be changed (except for that new instance layer type, should we so desire). An instance array layer perhaps (named as such). Any instance on that layer can be retrieved or sorted because for that layer type, the array becomes available to us (any other layer they can still do as they please in the future).

AtlaStar commented 1 year ago

Eh, I still think that'd be problematic design wise for less clear reasons; Layers don't require you to only put one type of thing into them via code, unlike the room editor that is a bit more strict on what can and can't be stuffed into a layer "type."

Basically, this sort of change would require a much stricter type of layer to be used, which means it'd also require a lot more checks with the general layer functions. Internally this might even necessitate the need for sub-classing and virtual dispatch, which could cause the drawing of layers to be slower in general.

This is why I was saying that instead of an explicit/implicit sort of a layer, something involving the instance dictating whether it needs to do some sort of depth changing would make more sense.

So basically I am thinking something like this you can call in an instance you want to do special depth sorting for

layer_instance_depth_test_layer(layer_id or layer_id_array,  callback_fn)

All this would do is check to see if the instance is intersecting with assets on the provided layer/s, and use the callback function to determine how to change the layer it is on by passing the offending assets positional data, and adding the return to that layer's depth. So something like depth = -y would just look like

function(element) {
  return (element.y - y)
}

This would put you on a higher depth value relative to the layer, if the players y is greater than the offending assets y value, which will happen when the player is further up the screen than the asset just as depth = -y does. Then if an intersection isn't occurring, it could just put you back on the layer it was previously on. The function would just set a depth value for your instance, so you can check if it is in a managed layer or not that way, and it would automatically put the instance back into the layer it was on prior to going onto a managed one. The only other addition that may be useful is probably to either always test against managed layers as well as the passed ones, or to have a boolean passed to the function as well that says that it should also test against the engine managed layers so that things like instances depth sort with each other when on managed layers due to also being behind something else as well.

The above logic could be implemented ourselves right now even, but the only aspect that'd differ is that we'd need to control the 'managed' layers ourselves and do some stuff to get the asset type from the element id's of things in that layer to do testing against, which will always be slower than something built into the engine.