Open jkeon opened 10 months ago
This is an interesting idea but I have some concerns. I've been working in this space quite a bit lately :)
I get that filtering your make work set is cumbersome but do we really want to pay the allocation cost of a NativeList(Allocator.TempJob)
and its associated bucket allocations?
Unless the comparison is really expensive you're probably better off just skipping over entities that don't meet your filter in the job itself.
To reduce boilerplate we may be better served by a custom job type that takes in filters and automatically does the skipping for you. I may already be close to a solution for this with my custom enumerator that takes in a filter in QuestCollection
of our project. We can improve it further by making it chunk aware and striding over the chunk of filtering on the existence of a component (Tag) or the value of a SharedComponent
.
Sort worries me because you essentially turn your data set into a series of random lookups after you sort it.
If you're just looking for the highest value or highest X values then you can track those as you iterate instead. Lots of boilerplate today but probably something we could create some convenience patterns for.
If you truly do need all of your results sorted (Ex: Some UI presentation) then you're best served to do the sorting after your filtering and any other processing/data collection you want to perform. The native collections have async and sync sort functions to help you with this today.
I think we're close to this today. LazyReactiveValue
is most of the way there and just needs to support async scheduling its refresh when an Acquire is requested on the data. Very doable.
Using vanilla Unity
EntityQuery
objects are powerful for getting the subset of data you want but they are limited to the structural aspect of the archetype.Example: Get all Containers that are full
From a query perspective, finding anything with these components is going to be super fast and return only the entities that are containers and are full.
However from a practical perspective, this implies some really inefficient practices. Whenever we add to the Quantity, once we hit the max, we would need to add the
IsFull
tag which results in a structural change. We could change theIsFull
tag to a bool component that all Containers have but at that point we might as well just get rid of it all together and instead just compare the Quantity to the MaxQuantity to determine if a container is full or not.We're efficient from a structural change perspective and storage usage, but we're offloading the mental load and coding for jobs to the developer who only wants to operate on
Full Containers
.With
Entities 1.0
onUnity 2022.2+
the job scheduler has reduced a lot of it's overhead. See: https://blog.unity.com/engine-platform/improving-job-system-performance-2022-2-part-2Which allows for an architecture of scheduling more micro bursted jobs to make sense.
This could allow for us to write our own extension methods to scheduling jobs that allow for better quality of life Queries and less boilerplate.
LINQ Style Syntax
Relationships are hard in ECS. Flecs author estimates it will take 2 years for him to build it out properly. https://ajmmertens.medium.com/a-roadmap-to-entity-relationships-5b1d11ebb4eb If we had this in Unity today, then the Query system would allow for it, but we don't, so we have to make due with pre-jobs.
With Unity's Burst and deferred
NativeList
we should be able to string together micro jobs that can whittle down the Archetype set into the set that we actually care about for our heavy lifting jobs.Which would result in the end user code looking like:
Options
FilterOn
IsContainerFull
would want to get theQuantity
andMaxQuantity
components from the archetype query.IsContainerFull
would check if theQuantity
is >=MaxQuantity
and return true or false to allow for the entity to make it through to the next part.SortOn
ToResult
Quantity
orMaxQuantity
, we just cared if the Container was full. What we care about in our job is theEntity
.DoSomethingWithFullContainersJob
will get triggered and run on this final result list ofEntity
objects.ToResult
job struct that would pull in whatever was needed to generate the result struct.Implementation
Possibly need to use SourceGenerators which would build up the pre-jobs and ensure we're pulling in the right component lookups to build the queries.
Might be able to get away with it via generics but we might get a monstrosity that looks like this:
Would want to make sure that we cache the results and only re-run the pre-jobs if we need to.
For example, we have a job that runs near the start of the frame on full containers.
Quantity
orMaxQuantity
written to so the cached result from the beginning of the frame is still valid. No pre-jobs needed.Quantity
This approach "just works" but you could run into issues where you're running the prejobs way more than you need to. We should issue a warning to the developer or some way for them to know that they MIGHT want to look at ordering better to make use of the cached results.