search

rust-lang / rfcs

RFCs for changes to Rust
https://rust-lang.github.io/rfcs/
Apache License 2.0
5.96k stars 1.57k forks source link

Iterator::scan too limited #1729

Open crlf0710 opened 8 years ago

crlf0710 commented 8 years ago

Surely for the simplest case, it works. However, the unfinished state may contain extra data that may hint us generate one or more Bs.

I believe the more sound way is call f for each item wrapped with Some(), and then call f with None until it yields None...

bluss commented 8 years ago

The issue is a bit unclear to me. Can you make a small example?

crlf0710 commented 8 years ago

Suppose i want to write a tokenizer, that is, split a string into an array of strings containing each word. One possible thinking is that store the unfinished word into the state and send it out when it is finished. However for the last item, there's never the chance to do so, and the unfinished word inside the state get lost.

bluss commented 8 years ago

I see.

We can't change the stable Iterator::scan at this point. You're welcome to suggest a new iterator adaptor that would work better.

Be sure to look inside itertools and other places in the crates.io package ecosystem. Those can host your new adaptor with a lower bar of entry, presumably.