Open yoshuawuyts opened 3 years ago
Is there a plan to include a core::stream::StreamExt
(similar to the one in the futures crate) as well?
Is there a plan to include a
core::stream::StreamExt
(similar to the one in the futures crate) as well?
The plan is to add methods directly onto Stream
much like methods exist on Iterator
, but we want to do so in a way that won't cause ambiguities with ecosystem-defined methods in order to not accidentally break existing codebases when upgrading to newer Rust versions.
Some discussion on Zulip raised the question of naming. With full acknowledgement to the fact that the name Stream
has a long history in the async ecosystem, multiple people observed that something like AsyncIterator
or similar might be much more evocative for new users. Such a name would allow people to map their existing understanding of iterators. "I understand Iterator
, and I understand async, and this is an async version of Iterator
".
As a new, inexperienced user, I find AsyncIterator
way more foreign than Stream
, to be honest... perhaps I'm not too involved in compiler discussions and I don't see how the jargon clicks together though :-S ... also, blogposts are already being written about Stream
, so changing the name mid-flight will only breed confusion, I reckon?
Prior art on "iterator" and "async iterator" naming schemes in other languages:
Symbol.Iterator
, Symbol.AsyncIterator
IEnumerable
, IAsyncEnumerable
__iter__
, __aiter__
Sequence
, AsyncSequence
Context: We're prototyping some simple CLI app functionality. We're following the mini-redis example code where we can.
We've bumped out head on streams, and the need for the crates async-streams, and parallel-streams.
In our experience some *Iterator
terminology would have helped clarify what streams are.
Given the need for the crates we cited, especially the async-streams RFC, we wonder if there isn't a need for:
Iterator
ConcurrentIterator
ParallelIterator
Or is the intention that async-stream and parallel-stream crate efforts all able to converge into a stream
that covers the concurrent and parallel use cases?
Rayon provides a whole ecosystem of parallel iterators on top of a work-stealing threadpool, and is currently the de facto Rust standard for parallel iteration. But I don't foresee a parallel iterator trait getting into the Rust standard library anytime soon.
Streams are supposed to cover the use case of concurrent iterators only (per my understanding). Hopefully once streams are stabilized into the Rust standard library we can have some syntax to use them in concurrent for loops just like we currently use synchronous iterators in for loops. However, there are still some issues to work out like what to do if a a stream panics or is dropped. Settling on a name (Stream vs AsyncIterator vs ConcurrentIterator) will be the easy part! :stuck_out_tongue_winking_eye:
Unfortunately, it's difficult to combine parallel and concurrent iteration at the moment. This would require Rayon to support a way to move tasks from worker threads to an async executor thread, or for an async executor like Tokio to support parallel thread pools in a more sophisticated way than tokio::task::spawn_blocking()
. Until that happens, most programmers try to get all their data into memory first on a concurrent executor-driven threadpool and then offload the synchronous computation to a parallel threadpool (e.g. managed by Rayon).
multiple people observed that something like
AsyncIterator
or similar might be much more evocative for new users. Such a name would allow people to map their existing understanding of iterators. "I understandIterator
, and I understand async, and this is an async version ofIterator
".
I've filed a PR to the RFCs repo, updating the "streams" terminology to "async iterator" instead: https://github.com/rust-lang/rfcs/pull/3208.
why did .next
make it. where can i get more info on this ?
One potential concern re: methods on AsyncIterator
, which is less of a concern and more of a justification for requiring them, is methods that use internal versus external iteration.
For example, it can be very efficient to perform a fold
on an iterator which is composed of several chain
s, but it can be much less efficient to directly call next
on said iterators.
My concern is that without parity between the methods on Iterator
and AsyncIterator
, much of these optimisations that already exist for Iterator
will be lost in the conversion to AsyncIterator
. Since, as it stands, there's not really a way to run a for_each
or fold
on a regular iterator if the loop contains async operations.
I know for a fact that the current async ecosystem with futures::stream::Stream
does not have parity with Iterator
, with some notable surprises including the fact that try_fold
requires the iterator item to also be composed of results, rather than just the return type of the function.
It would be extremely detrimental to the idea that "this is just the async version of an iterator" if there weren't parity there IMHO, since users might notice slowdowns in code that simply calls std::async_iter::from_iter
without adding any extra async code at all.
@clarfonthey yes, definitely. The concerns you raise are valid, and the working group is currently actively investigating how we can ensure parity between sync and async Rust APIs. We don't yet (but should) have guidelines on how methods on async traits should be translated from sync to async, but that likely needs us to land async closures / async traits first.
Given the current trend of the async working group, I wouldn't be surprised if this can become async fn next
rather than fn poll_next
.
@joshtriplett ah yes, that's definitely something we've been discussing within the working group, and we're currently working towards enabling that. The other thing we're currently researching is keyword-generics, which may allow us to merge the separate Iterator
and AsyncIterator
traits into a single Iterator
trait which is generic over "asyncness".
I'll update the tracking issue to reflect both these items.
(NOT A CONTRIBUTION)
Given the current trend of the async working group, I wouldn't be surprised if this can become async fn next rather than fn poll_next.
ah yes, that's definitely something we've been discussing within the working group, and we're currently working towards enabling that.
I think this is not the right direction for this feature. I hold this view very strongly. AsyncIterator::poll_next enables library authors to write explicit, low-level async iteration primitives for unsafe optimizations. Relying on the compiler generated futures of an async function will make the layout of these types much less predictable or controllable and will take this away from users.
For ease of use where this fine-grained laying control is not desirable, generators are the play (async or otherwise), rather than having to write next methods (async or otherwise) at all.
You need to have the low level APIs (Future::poll, Iterator::next and AsyncIterator::poll_next) for hand-rolled optimized code that the compiler can't be relied on to generate. You need to have the high level syntax (async functions, generators, and async generators) for when people just want to get things done and don't care about these kinds of optimizations. You need both. An async next method would be doing each side of it only halfway (low-level iterator, high-level asynchrony), and would basically trap Rust in a local maxima that looks appealing from where we are now but would not be the best final state.
EDIT: What I mean when I say that "generators are the play" and "local maxima" is that I think because Iterator::next has always existed and has a superficially simple API (ie no Pin, no Context), its not as obvious that for ease of use implementing an iterator with a next method is actually an awful experience for users. Yielding from generators would be much easier. So when you want the ease-of-use story, you want generators, and those can be made async just as easily as functions can. But generators can't guarantee the representation that gets the codegen from for loops over slices looking so good, and similarly won't guarantee the optimizations some async code will want as well. You should be thinking of implementing Iterator::next as really as low-level as implementing Future::poll.
EDIT2: I think the counterargument to this is that mixing-and-matching high-level and low-level is also desirable. IE next + async is desirable when you want control over iteration but don't care about control over asynchrony. Analogously, there must be a hypothetical API that's control over asynchrony but compiler generated iteration - a polling generator(??). I think there could be a case to be made that users do want to be able to drop down into fine control over one aspect of their control flow but not the other, but then that should be an additional, third (and fourth?) option in addition to full control or full ease of use, you can't get rid of the full-control option, which is poll_next.
For the people that haven't closely followed along on the "blogosphere", I'll link to a few excellent blog posts about it (in chronological order) (authored by people in this thread) (by no means exhaustive):
Another argument in favour of fn poll_next
that I've not seen explicitly mentioned: we could still provide a default async fn next
impl, to somewhat improve the user experience of manually calling next
:
async fn next(mut self: Pin<&mut Self>) -> Option<Self::Item> {
poll_fn(|cx| self.as_mut().poll_next(cx)).await
}
// Or
async fn next(&mut self) -> Option<Self::Item>
where
Self: Unpin,
{
poll_fn(|cx| Pin::new(&mut *self).poll_next(cx)).await
}
Certainly, this is not as clean as the simple async fn next(&mut self)
, see this playground link for an example of how it might be more verbose, because we're allowing the iterator to be self-referential, but might serve to strike enough of a balance?
Whether Rust then goes with self: Pin<&mut Self>
or Self: Unpin
mostly depends on what the plans are in the future for making Iterator
able to be self-referential, which I think is still an open question.
@withoutboats for your latest blog post, can you give an approximate loop desugaring like in https://github.com/rust-lang/rust/pull/118847#issue-2036784747 I'm not sure if I'm parsing the ascii charts properly.
Anyway, I'm reraising a concern here that I already mentioned in the PR and that's similar to clarfonthey's:
I think the current proposed interface is terrible for performance when iterating on small types (e.g. u8
s) because the poll_next
interface returns a Poll<Option<Item>>
that conveys two states at the same time, readiness and end-of-iteration.
If the loop body contains no await points, just munching some bytes for example, then that loop body would ideally optimize to a single induction variable based on next
's internals. I.e. just branching on "do we still have more data to process".
Only when reaching the end of available data it should poll. This also requires a separation of progress information and getting the next item(s), but it tries to solve a different problem than boat's proposal.
One option is to make poll_next
not-async (basically just next
) but only return items when the iterator has made progress, which would be polled by a separate method. poll_progress
would then return a Poll<bool>
I guess to indicate more items / end of iteration.
Another approach is returning an I where I: IntoIterator<IntoIter=IN>, IN: ExactSizeIterator
from poll_next
. Option
fulfills these bounds but an async iter that has an internal buffer can instead choose to return some iterable with more than one item, which allows the loop body to process them on a single induction variable of that iterable without polling.
My async understanding is limited, I'm coming from the sync Iterators side. So I may have misunderstood something.
Given the current trend of the async working group, I wouldn't be surprised if this can become
async fn next
rather thanfn poll_next
.
After rust 1.75 released, trait async func is stable, async fn next
is more user friendly.
Given the current trend of the async working group, I wouldn't be surprised if this can become
async fn next
rather thanfn poll_next
.After rust 1.75 released, trait async func is stable,
async fn next
is more user friendly.
In the rest of the thread, withoutboats has argued that we need async generators for ergonomics, rather than async fn next()
, which is less fine-grained than fn poll_next()
and less ergonomic than async generators in their opinion. Then, what is your rationale for promoting async fn next()
over async generators?
Looking at this through the lens of algebraic effects, I would say that
Future
corresponds to a (suspended ongoing invocation of a) function that can trigger the "pending" effect. The effect has argument and return type ()
(and eventually the function returns Future::Output
).Iterator
corresponds to a (suspended ongoing invocation of a) function that can trigger the "yield" effect. The effect has argument type Iterator::Item
and return type ()
(and eventually the function returns ()
).This analogy is not quite perfect: futures can be polled again after returning "ready" (and similar for iterators); futures have this "context" argument; iterators are not pinned. But I would argue all of those are concessions to how we model these kinds of computations in Rust -- the abstract concept we want to model doesn't have them, but the imperfect realization of that concept in Rust does have them.
An AsyncIterator
would then be a (suspended ongoing invocation of a) function that can trigger both the "pending" and the "yield" effect, with the same types as above (and eventually the function returns ()
).
We don't have algebraic effects in Rust, but futures have shown how we can model one particular algebraic effect. If we follow that same paradigm, then the fn poll_next
-based encoding seems to be the most direct way to represent an AyncIterator
.
In contrast, the async fn next
-based encoding seems to model something different: it represents a function that can trigger a "yield" effect, where the argument type of this effect (i.e., the data being passed from the function to the handler) is a function that can trigger a "pending" effect. If we view an async iterator as triggering a sequence of yield and pending, then this does seem equal in abstract expressivity to fn poll_next
(split up the sequence after each yield, to obtain a sequence of subsequences; then each of these subsequences corresponds to one of the functions that is being yielded). However, it is a much less direct encoding of what actually happens, involving a seemingly unnecessary "thunk" (the functions being yielded). Sure, in trivial cases this can be optimized away, but I wouldn't bet much on the claim that such optimizations will always work. It certainly does not seem to fit the usual Rust philosophy of avoiding unnecessary overhead in the most basic abstractions.
So I guess what I am saying is, I tend to agree with boats. Mind you, I'm not an expert in async, I am taking a 10,000 foot view of this problem.
speaking of https://doc.rust-lang.org/nightly/std/async_iter/index.html#laziness --- can the basic async iterator has a usage example? right now these docs show how to define the counter and implement the async iterator trait, but not how to actually invoke it or use it as an end user.
is this because to run this async iterator we need externals like tokio? if so, how do we ensure std has what it needs to run async iterators without needing an external runtime
here's the part i mean, the code isn't really showing what to do with the counter, so it's hard to look at this page of rustdoc and know fully how to get rolling with a minimalist async iterator, which tbh is likely to be much more rewarding for all of us than futures, just a hunch, async iterators are the move, because they promote categorical thinking [about i/o] and especially for input streams
seems like a no brainer to have the example here be a perfectly optimized copypasta for std-only async iterator over some particular user defined type of messages from a generic tcp stream output buffer of bytes, for example
as an aside, i wonder if, in the context of async, some subtle fundamental limitation in Poll<T>
causes Pin<&mut self>
Which makes me wonder if
pub enum Ratchet<A, B> {
Pending(A),
Ready(B),
}
Could give us a different design avenue for async/await besides Poll<B>
if I make sense.
Ratchet<A, B>::try_forward
could be a better option than Poll<B>
with Pin<&mut A>
because
the Ratchet::Pending(A)
can take the place of Pin<&mut A>
and be potentially better aligned with necessity to preserve referenceability of the owned pending state (giving it the term "A" instead of the disembodied "self" on some other detached thing)
but maybe
Ratchet<A, B>
is too tightly coupled between the "self" of the "Future" (A) and the Future::Output
(B)? just thinking out loud, sorry, carry on
@rustbot labels -I-libs-api-nominated +T-lang +WG-async
This was discussed briefly in the libs-api call today. This work is going to be driven from the lang side on the basis of work done in WG-async, as we had discussed in the meeting on 2024-05-14. On that basis, the decision was to unnominate.
meeting on 2024-05-14
Where can I find the minutes of that meeting? In the t-lang/meetings Zulip channel, I can only find the meeting on 2024-05-15, but that was about a different topic (match ergonomics).
EDIT: Oh, that was a libs-api meeting, not a lang meeting. oops
This is a tracking issue for the RFC "2996" (rust-lang/rfcs#2996). The feature gate for the issue is
#![feature(async_iterator)]
.About tracking issues
Tracking issues are used to record the overall progress of implementation. They are also used as hubs connecting to other relevant issues, e.g., bugs or open design questions. A tracking issue is however not meant for large scale discussion, questions, or bug reports about a feature. Instead, open a dedicated issue for the specific matter and add the relevant feature gate label.
Steps
Unresolved Questions
Stream
andIterator
https://github.com/rust-lang/rust/pull/79023#discussion_r538975223Iterator
, clarifying panic behavior. The panic behavior betweenStream
andIterator
should be consistent.Stream::next
. This was removed from the RFC because it prevents dynamic dispatch, and subsequently removed from the implementation. This should be resolved before stabilizing.fn poll_next
toasync fn next
once we can useasync
in traits.Iterator
andAsyncIterator
into a single trait which is generic over "asyncness".AsyncIterator
instead? https://github.com/rust-lang/rust/issues/79024#issuecomment-816868752Implementation history
core::stream::{Stream, Next}
Stream
toAsyncIterator