kpreid / yield-progress

Progress reporting and cooperative task yielding in Rust
Apache License 2.0
2 stars 0 forks source link

Concurrent task progress combiner #20

Closed kpreid closed 2 months ago

kpreid commented 4 months ago

There should be an operation which splits a single YieldProgress like split_evenly() does, but can be used concurrently rather than sequentially.

This adapter will need to perform a heap allocation, to remember the child information and combine it as individual children send updates. Ideally it only requires alloc and not std, but I haven't thought about whether the synchronization requirements can be met that way. (Some applications might prefer that the combination work be done in a separate task rather than all children, but that would involve creating a channel and all sorts of policy decisions; those applications can implement their own split.)

Labels will be tricky; perhaps they should be combined using a user-provided function, since we cannot know whether showing a few or all of them is more desirable.

kpreid commented 3 months ago

21 implements this, but I want to give the feature a trial run in a real application, before releasing it, before closing this issue.

kpreid commented 3 months ago

It seems to work well enough, but I think I want to allow the split to be weighted. (The analogy with split_evenly() is imperfect because split_evenly() can be remade out of simpler split()s, but each split_concurrent() has a cost of creating the combiner.

kpreid commented 3 months ago

Another thing that might be of interest in the general design of splitting is to provide a label for each child up-front rather than as a later mutation. Are there enough things that it makes sense to provide a fluent/builder sort of splitting?

kpreid commented 2 months ago

26 renamed split_concurrent() to split_evenly_concurrent(), and this feature as it is so far is released in version 0.1.6. Further improvement can be separate issues.