Feature request: Optional feature for rayon support

[nothingnesses]

Currently, code like the following don't work:

use bitvec::prelude::*;
use rayon::prelude::*;
use std::sync::atomic::AtomicUsize;

fn main() {
    let mut list = bitvec![Lsb0, AtomicUsize; 0; 100];
    list.par_iter_mut().step_by(2).for_each(|mut bit| {
        *bit = true;
    });
    println!("{:?}", list);
}

But the above works if vec![false; 100] is used instead of bitvec![Lsb0, AtomicUsize; 0; 100] (or bitvec![0; 100]), or if .par_iter_mut() is replaced with .iter_mut().

It would be great if bitvec had support for rayon to allow its data structures to be iterated through in parallel in a convenient manner (i.e., with rayon's par_iter() or par_iter_mut() methods). Afaiu, this would require rayon's ParallelIterator and/or IndexedParallelIterator traits be implemented for bitvec's types.

Fwiw, here is part of a message from @myrrlyn about one way support might be added:

rayon methods require that i directly implement its extension traits. they don't have a blanket impl Iterator -> impl ParallelIterator transform. you can fake it with a much uglier boilerplate by using rayon::scope to drive a serial iterator, then parallelize it by making the loop body spawn a thread containing the rest of the work onto the rayon scope

[myrrlyn]

Sample program that currently works:

use bitvec::prelude::*;

#[test]
fn parallelism() {
  let bits = bits![mut Lsb0, usize; 0; 800];
  rayon::scope(|s| {
    for (idx, chunk) in bits.chunks_mut(20).enumerate() {
      s.spawn(move |_| {
        chunk.store(!0u32);
        println!("{:02}", idx);
      });
    }
  });
  assert!(bits.all());
}

Implementation notes from doing a quick sketch:

• Rayon is a good way to test that I have correctly managed impl {Send,Sync} in bitvec; should've used that much sooner
• Rayon defines new slice par-iter adapters. I also will need to do this, because implementing Rayon's iteration traits directly on my seq-iter adapters causes method resolution failures. Bless up for macro_rules!.
• My implementation should pretty much just copy directly from rayon/src/iter/plumbing/mod.rs and rayon/src/slice/mod.rs. Bridging to the thread-pool is Rayon's problem, not mine, and they've solved it. The only real logic is in <MyType as Producer>::split_at; everything else appears to be boilerplate for gluing types together.
• This appears to be fairly straightforward, but it's going to be a lot of tedious work putting everything together. Certainly doable, equally certainly not something I can afford to front-load unless requested.

Questions:

• Rayon only defines these parallel slice iterators. Should we define par-iters for everything in bitvec/src/slice/iter.rs or just the Rayon ones? Reasons against: it's extra work that they didn't do, so by definition, [bool].rchunks(W).par_iter() doesn't work and therefore nobody can expect BitSlice.rchunks(W).par_iter() to work. Reasons for: bitvec is already a superset of core, so there's precedent for going above and beyond the source.

I probably won't get back to this until, oh, February, so this should serve to restore everything I learned today when I do pick it back up.

[nothingnesses]

It seems rayon has added implementations for RChunks, although there hasn't yet been a version published on crates.io containing these changes.

[bonsairobo]

I found it easiest to just use bits.as_raw_mut_slice().par_iter_mut() and manually set bits of the integer storage.

[Anivie]

Is there any progress now? Can we implement traits such as FromParallelIterator<bool> to facilitate more convenient collaboration with Rayon?

let mask = [10.; 100];
let m = mask.par_iter().map(|x| *x > 100.).collect::<BitVec>();

[tcerqueira]

Is there any progress now? Can we implement traits such as FromParallelIterator<bool> to facilitate more convenient collaboration with Rayon?

let mask = [10.; 100];
let m = mask.par_iter().map(|x| *x > 100.).collect::<BitVec>();

This one has been bugging me, I can iterate in parallel but collecting I'm not sure. Maybe collecting into the underlying T and then call BitVec::from_vec?