yoshuawuyts / futures-concurrency

Structured concurrency operations for async Rust
https://docs.rs/futures-concurrency
Apache License 2.0
400 stars 31 forks source link

How should I factor my specific problem, and more broadly, how should I handle `Result<!, E>`? #152

Closed bradleyharden closed 1 year ago

bradleyharden commented 1 year ago

Hi,

I've been using the tokio join and select macros in some tests of mine. I recently came across this crate, and I like the thought of moving away from macros to proper concurrency primitives. However, when I tried to fit my tests into the existing options in this crate, I can't seem to make it work without adding some additional logic that is ultimately unnecessary. I thought I would post my situation here, in case I'm either missing something or it's a genuine use case that could use its own primitive.

In my case, I have three tasks, call them A, B and C. With a successful test, I expect both A and B to return Ok, but C doesn't actually need to return; it can just be dropped. If there is a problem, any of the three could return an Err.

In this situation, neither Race nor RaceOk is acceptable, because I need to confirm that both A and B returned Ok, but I also can't use Join or TryJoin natively, because it will hang if the test is successful. I could maybe use Merge, since they all return anyhow::Result<()>, and I could verify that I get exactly two Ok, but that feels like a hack.

My current solution, and the solution I would use with futures-concurrency, is to add a way to gracefully shutdown C by giving it a CancellationToken. In my specific case, that's not too much work, because C is actually just a loop with a single await point. But ultimately, the graceful shutdown of C is unnecessary, and the added logic only clutters the code. That clutter would also be worse if C were more complicated.

Now that I think about it more, I suppose I could TryJoin A and B and then Race that future with C. I think that would work, but it wouldn't return all of the errors. It would only return the first. I suppose that's an inherent problem, though, since I can't guarantee all tasks will complete in the event of an error somewhere.

Is this a situation you've seen before? Is what I came up with just now the "correct" implementation in your mind? More broadly, what do you do if you expect some future to never return unless there is an error? Essentially, how do you handle Result<!, E>?

yoshuawuyts commented 1 year ago

Heya! This is a fun question, thanks for asking!

In this case, can C succeed with anything other than an error? If it can only ever fail, then it's easy: try-join A and B, then race that join with C:

let ab = (a, b).try_join();
let f = (ab, c).race().await?;

If C can also return with a success, then you're indeed right and we need to provide a way to short-circuit execution of C. One way of doing that would be to try-join A and B again, and then on completion of the join drop the sending side of an async-channel channel. This will cause the receiving side to resolve, which will close the channel. If you race task C with the receiver you now have your cancellation token done:

use async_channel::bounded;

let (sender, receiver) = bounded(0); // zero-sized channel
let ab = (a, b).try_join().map(move |res| {
    drop(sender); // both a and b completed, cancel c
    res
});
let c = (c, receiver).race();
let f = (ab, c).try_join().await?;

You'll probably need to align the return types for all of this to work, but I hope one of these approaches will work for what you're trying to do!

bradleyharden commented 1 year ago

If it can only ever fail, then it's easy: try-join A and B, then race that join with C

Yeah, I realized that as I was typing up my first post. There's really no better debugging tool than explaining your problem in detail to someone else.

In my case, C never returns in the successful case. It is indeed Result<!, E>. So TryJoin + Race is what I need. Thanks for taking a look.