outfox / fennecs

... the tiny C# ECS that loves you back!
https://fennecs.tech
MIT License
162 stars 5 forks source link

CROSS JOIN aliasing issue with certain Jobs #11

Open thygrrr opened 3 months ago

thygrrr commented 3 months ago

What:

Certain algorithms operating on CROSS JOINS, e.g. in N-Body Problem sample, break under parallelization (i.e. when being run as a Job instead of a For) due to write destination aliasing.

Prelim Fix: (todo in 0.5.x)

Job can throw before operating over Stream Types with Wildcards. This is trivial to implement in O(1) - in Query<>.Job> / Stream<>.Job(), check if any Stream Type is a wildcard and boom. 💣

This can even be paired with the below "wait for iteration WorkItems to finish" for a great halfway fix; the proper fix however mitigates the impact of Wildcards if the Archetype actually doesn't match them more than once - which is very common.

Proper Fix: (circa 0.7.x)

Join can likely report not only that is it empty or populated, but also whether it has repeat enumerations of the same storages (a simple bool - I think this can be calculated in advance for the entire join before the first Select/Iterate())

Then, Query.Job (or Stream.Job) needs to not queue new WorkItems for the next Iteration before all previous Items have completed (but it can still queue all chunks of each individual iteration at once, because there's no possibility of aliasing between WorkItems in the same Select!)

The problem arises when Join.Iterate() basically swaps Storages around for repeat enumeration, and many patterns of N:N enumeration can be problematic here if multi-threaded.

Related:

This may or may not become a different / simpler issue when other write aliasing protections are in place (e.g. Microscheduler for multiple parallel Jobs)

thygrrr commented 3 months ago

Preliminary fix was made in 0.5.0 or 0.5.1, with the AssertNoWildcards placeholder in the job functions.