Feature Request: Threadpool returning sortable vectors

[Beherith]

In my mind, it sounds like it could be a fun addition to have a for_mt function that would take struct vector type argument, and do work on some iterable, and return the results of each thread in a separate vector. Optionally, as a last step, each thread could sort its own vector, with the final mergesort of all vectors of each thread being hopefully less taxing given each thread is already sorted.

Here is the engine implementation of the current for_mt stuff: https://github.com/beyond-all-reason/spring/blob/BAR105/rts/System/Threading/ThreadPool.cpp

In my simple mind, it would work like this:

[lhog]

What kind of vectors (and their sizes) do you plan to feed to that future for_mt like function, so they need to be MT-sorted for efficiency?

[Beherith]

The main use case would be to create depth-sorted renderbuffers in an MT way. But I realize a shortcoming of my description above: It doesnt address the case where the MT implementation would do a 'filter' type operation, where, for example, we spread all particles out to threads, and within those thread decide if the particle needs to be drawn. If it does need to be drawn, then it could return a struct of floats for renderbuffer submission. But I did not think about the case where it shouldnt be drawn, and thus no data needs to be returned.

[lhog]

Got it.

Also once upon a time I had a per-thread RBs, so projectile->Draw() calls could be MTed (and were MTed), these would later be merged into a common RB and sorted/drawn accordingly. Maybe my measurements were bad (it was before Tracy times), but back in the day I concluded no perf improvements were achieved, so the code was removed without traces. Happy to be wrong on this one though.

[Beherith]

Things are indeed very different since Tracy is available :D It gets better every day