search

rust-itertools / itertools

Extra iterator adaptors, iterator methods, free functions, and macros.
https://docs.rs/itertools/
Apache License 2.0
2.72k stars 309 forks source link

Efficient flattening of Vec<Result<Vec<T>, E>> into Result<Vec<T>, E> #817

Closed mo8it closed 9 months ago

mo8it commented 10 months ago

See my post on the Rust forum with benchmarks of process_results in itertools: https://users.rust-lang.org/t/300x-more-efficient-flattening-of-vec-result-vec-t-e-in-the-worst-case/103841

Philippe-Cholet commented 10 months ago

Well, I think itertools adaptors are convenient first, not necessarily the fastest in every situation. We are working on specializing fold methods (see #755 and related PRs) to improve on that front, and we would specialize try_fold if we could do it on stable Rust.

If you think that specialize a method of our ProcessResults is worthwhile, then it would surely be a nice addition. Or maybe another method/adaptor?

Philippe-Cholet commented 9 months ago

A function taking a vector can leverage all its information which process_results simply can't do as it handles an iterator of results, but it's not supposed to do it as it would stop at the first error, which a vector would not do.

Those are two solutions for a similar problem but with different trade-offs. It's worth knowing but here is more about iterators than vectors, results or flattening so I close this.