[Feature request] Support views filtering by absent components?

[BBI-RamonZarateSaiz]

Is there any reason why entt does not support defining views by being able specify missing components as well as existing components?

So for example I could iterate all entities that have a position component but not an orientation component.

I find it a very useful feature when working with ECS, so I wonder if this has come up before (did not find anything) or if there is any technical reason this could not be done, or any other reason.

Thanks!

[indianakernick]

I suggested this a while ago. A few of us had a discussion about it on Gitter. I can hardly remember the conversation but I think our main concerns were that even the best compilers will struggle to turn the crazy templates into code that is as fast as an if in the loop body. The feature made it into the TODO list and we left it there.

Although, what I was suggesting was doing complex things like A or B, any 1 from [A, B, C, D], not A. If you just want to exclude components then the templates won't be as crazy and it will probably be just as fast as an if in the loop body.

[BBI-RamonZarateSaiz]

Thanks for the reply! Yeah I thought it was weird it had not come up.

I can see the performance being a potential concern, but at the same time I would expect excluding components is quite comparable (check en entity is in a set vs check an entity is not in a set). Probably things get more expensive for persistent views?

But for example if the perf was comparable to an if, then that would be overall a gain IMO since right now I need an if on a large query to filter out entities that have a component to be excluded. If the view excludes those I don't need to check the if for the ones I already know are on the view.

Performance aside, the resulting code is considerably cleaner in many cases, from my experience, so it would likely be worth it.

[ghost]

Minor input on the topic: The current iteration of Unitys ECS implementation allows so called EntityArchetypeQueries which fetch all entities adhering to the specified constraints of the following form:

new EntityArchetypeQuery
{
    Any = Array.Empty<ComponentType>(),
    None = Array.Empty<ComponentType>(),
    All = new ComponentType[]
    {
        typeof(PlayerController),
        typeof(Position)
    }
};

It allows to filter entities by required, optional and disallowed components.

[skypjack]

Is there any reason why entt does not support defining views by being able specify missing components as well as existing components?

Some reasons in random order:

• I don't need it and therefore I didn't implement it yet. Usually I flip the problem on its head by defining the concept of not having something through a component to assign to the entities that do not have what I'm looking for. It's a bit counterintuitive, but iterations are definitely faster this way.

• Because of how standard views are defined, the best they can do internally is a sort of if for each and every entity they inspect. That is exactly what you would do as a client of a view. The difference is that for you it's really only an if, while for a view it's probably template machinery that would end up being slower than an if.

• I put a note in the TODO list for future improvements, but to be honest I had other priorities so far and it's still there.

• AOB

That being said, I'm working on a great improvement for persistent views (if you are in the gitter room, you followed the discussion probably).
While working on it I realized that we can add all types of filters on these views without affecting performance during iterations.

Not sure yet, but likely you'll see support for filters on persistent views finding its way on master in future.
Unfortunately I've not an ETA for the feature. It mostly depends on my free time at the moment and I'm quite busy this month.

---

Side note for the whole bbi-* team. :-)
Do not forget to star the project if you like it and you use it!! Consider also supporting the development if possible, it would be really appreciated.

Thank you.

[skypjack]

@NearAutomata I thought about your comment about Any, None and All.

At a first glance:

• All is already there, namely the views.
• None is hard to do with standard views unless we put ifs all around, but it wouldn't make much sense to do it using a bunch of ifs. On the other side, persistent views could do it without ifs and probably it makes sense to allow it only on optimized views.
• Any is meh. The same reasoning as above but then we have ifs during the each, because the view cannot say if an entity is there because it has A or B and therefore it must query both of them.

If I had to develop something along this line, I'd proceed as it follows:

• Add support for none to the persistent views and that's all for what concerns views in general.
• Add an external tool (Group? Tracker? SuggestMeANamePlease?) built on top of a sparse set that allows for all the filters (all, none and any) and returns only entities. Users will know what to do with those entities then.

Keep in mind that the latter isn't a high performance tool, but I don't think we can really design something with the best performance and the strangest combinations of filters, so who cares?
It will serve the purpose on non critical paths, mostly gameplay (that is the sole point where I see uses for something like Any after all).

What about?

[vblanco20-1]

I dont see any need for "Any" at all, with the design of this library. It exists for unity becouse you can do "block iteration" with that, and do your "if" at the block level (few hundred entities at a time). Given Entt doesnt have that, you are better just checking "has" yourself.

For the negative views, i think its completely fine if they are on persistent only. After all they are mostly a "performance" construct. The idea is to do something like filter out "Static" objects from a physics system, or other similar things. You can of course do the same with "has", but it would be much slower. Having them do the "if" themselves inside defeats the point, i think. At that point its better if the user does the "has" himself.

[OvermindDL1]

Knowing 'any' can still make parallizing the work easier as then each system would know what is being accessed should that point come.

[skypjack]

To be honest, I never had the need for any and that's why I'm quite doubtful about investing time on it.
I cannot even imagine an use case. The sole that came up to my mind is something like - any of state 1, state 2 or state 3, to elaborate all the entities that own a state from a given fsm. Btw, being states exclusive, one can safely iterate them separately, thus any is pointless here.

Some examples that would convince me it's an useful feature?

[OvermindDL1]

Back in my system I had 'all' (all components were required for this system), 'optional' (any?, these are components that might be accessed in this system), and 'none' (these components must not exist).

The optional was useful because at load-time a dependency-graph of systems was built up and they would execute across threads for any with no all/optional overlap, the optional was mostly a marker for this purpose.

[skypjack]

@OvermindDL1 your optional wasn't the same of any, if I got it.
Here all/none/any are meant as - give me all the entities that have all of A, B, then none of C, D, then any of E, F.
So, you end up with entities that have either A, B, E, or A, B, F, .

[OvermindDL1]

So any here is more like xor rather than optional that I'd expect based on the name of any? I.E. either E or F is required, you must have one or the other? I could see that being useful at times but nothing immediately comes to mind, however a custom operation that we could pass in (a templatized thing ala mpl?) could be quite useful and generic for any use-case.

[skypjack]

@OvermindDL1 Almost. You can have either E, F or both of them actually.

[OvermindDL1]

Ah so it is an inclusive or then, so your example had 3 possibilities for the matching.

[BBI-RamonZarateSaiz]

For me, as long as I can exclude components, then any becomes unnecessary as now I can split the work between views. For example you can make a view to exclude E, one to exclude F and one to include both. You can then not use one of them depending if you want xor functionality or not.

[skypjack]

Note for the future me (out of some tests). Try with filter arguments with composable filters (functions).
It should work for persistent views. Not sure about standard views, but it's promising.

[skypjack]

I thought about this. At a first glance, there are two options with their pros and cons:

• Add filters (all/any/none) on persistent views. Views would return only components in all, for obvious reasons. The main drawback is that it affects construction and destruction of entities and components, as well as sorting functionalities. The API for persistent views would become more complex, but still usable. Filters cannot be attached on demand, they are fixed and different filters create different views. Systems won't have to store anything aside, so they can still be completely stateless.

• Create an external tool. They would return only components in all, for obvious reasons. They would affect only creation and destruction of components. The main drawback is that system cannot be anymore stateless, they must store these objects somewhere and cannot create them on the fly. Filters could be attached on demand, different filters don't create different objects.

Let's discuss this. Is having stateless systems a must have for which slightly worse performance during creation/destruction/sorting are the right price to pay?

[skypjack]

I made some experiments recently. See branch wip. Some notes:

1. This feature requires another signal to work properly. Consider something like this: all_of<>, any_of<>, none_of<A, B, C>. It won't work if you aren't notified about entity creation. The other way around is to store a sort of try_to_assign function along with the underlying pools and invoke it each and every time a new entity is created. Note that in both the cases this affects performance on entity creation to some extents.

2. In regard of the previous bullet, the sets of listeners strictly depend on the sets of component to observe (ie if you have an empty list as all_of, the set of listeners differs quite a lot from the case of a non empty list). This means more templates and sfinae'd functions/constexpr ifs all around so as to treat the different cases correctly.

3. Filters attached on creation/destruction of components are much more complex and greater in number. It means that such a feature will affect even more the creation and destruction of components under certain circumstances.

And so on. On the other side, benefits are arguable. Mainly because there are other if-less approaches to use to obtain the same result and EnTT already offers all what is needed for them.
Because of that, I don't think I'll work more on it in future. I won't even merge it on master any time soon probably. Reasons are:

1. As I said, the same result can be achieved using if-less additive strategies.

2. I don't need such a feature (because of point 1, that is the way I did it so far), therefore I prefer to avoid to develop and to have to maintain something I won't use.

3. Performance degrade a bit on some features and the codebase becomes more complex for no real benefits (it means that it doesn't allow you to do something you cannot do right now, it adds just another way to do it).

Unless someone pops out and gives me good reasons to proceed, I'll close the issue tomorrow as won't fix.

[skypjack]

As discussed with @vblanco20-1 on gitter, we can probably do something for the common case and just don't support all the others.
I reopened the issue so as to keep track of it.

In particular, something like this would work:

registry.persistent_view<A, B>(exclude<C>);

Put aside the syntax that will probably change a bit, these are the constraints:

• Available only for persistent views (it works also for standard views probably, still to look into them).
• Negative only views won't be supported.

Construction and destruction of components are still affected to some extents, but the boilerplate isn't that much.

Any thought?

[skypjack]

Please, take a look at the last commit (branch experimental). Feature still to be documented.

[skypjack]

Added a few notes about filters to the documentation. I'm almost ready to merge.
I'll wait for feedback for another couple of days, mainly because I'm out of home during the weekend. :smile:

[skypjack]

Merge on master. Time to close this issue.