rayon-rs / rayon

Rayon: A data parallelism library for Rust
Apache License 2.0
11.11k stars 501 forks source link

Implement non-consuming "_with" methods #545

Open sfleischman105 opened 6 years ago

sfleischman105 commented 6 years ago

Currently, ParallelIterator implements fold_with(), map_with(), for_each_with(), and reduce_with(). Each of them have a method signature similar to

fn ..._with<F, T>(self, init: T, map_op: F)
        where F: Fn(&mut T, Self::Item),
              T: Send + Clone

While useful for most cases, all the these operators consume the init: T parameter. I've run into a case or two where it's unnecessary to pass in the value directly, rather than passing in init: &mut T.

rayon::join can perform parallel computations with one side being a mutable reference, and the other being a cloned value. I think it might be fitting to allow these with operators to have a complementary variant where the value is taken in by mutable reference.

So, the additional function signatures would look like this:

fn ..._with_mut<F, T>(self, init: &mut T, map_op: F)
        where F: Fn(&mut T, Self::Item),
              T: Send + Clone

An example of this being useful would be for a minimax evaluation of a chess position.

fn minimax_evaluate(board: &mut ChessBoard, depth: u16) -> i16 {
    if depth == 0 {
        return board.evaluate_strength();
    }

    board.generate_moves()
         .as_mut_slice()
         .par_iter_mut()
         .map_with_mut(board, |b: &mut ChessBoard, m: ChessMove | {
             b.apply_move(*m);
             let score = -minimax_evaluate(b, depth - 1);
             b.undo_move();
             score
         }).max()
        .unwrap()
}

With the current map_with function, this is not possible without an additional board.clone() inside of the map function parameters. This results in unnecessary cloning for every iteration. And while I've currently implemented this in a more verbose form using rayon::join, these additional methods would allow for more flexibility and ease of use. Though I'm unsure of the safety of these methods.

cuviper commented 6 years ago

It would be nice if we could avoid proliferating too many variations of this. Can you achieve your goal by composing with other data structures?

I think something like MuCow might do the trick. Start with MuCow::Borrowed, then the clones will become MuCow::Owned, and they all implement DerefMut to give you the reference.

sfleischman105 commented 6 years ago

Using MuCow seems to work for my needs, thank you!

It might be worthy to explicitly mention that the _with iterators are consuming inside the documentation.