ThreadRng constructor with alternate seed source

[Pr0methean]

Background

What is your motivation?

To develop a drop-in replacement for rand::thread_rng() that works around https://github.com/rust-random/rand/issues/1357 by using a buffering wrapper around OsRng where the buffer is a channel shared across all threads.

What type of application is this? (E.g. cryptography, game, numerical simulation)

A library intended for multithreaded Monte Carlo simulations.

Feature request

At https://github.com/Pr0methean/shared_buffer_rng/blob/85cf1caf6ffbb7018ff6d4464f4fc7b07d34b3d6/src/lib.rs#L110, to create an otherwise-identical drop-in replacement for rand::thread_rng() that was to differ only in how it obtained seeds, I had to copy many of ThreadRng's implementation details:

#[derive(Clone, Debug)]
#[repr(transparent)]
pub struct ThreadLocalSeeder<const WORDS_PER_SEED: usize, const SEEDS_CAPACITY: usize, SourceType>
(Rc<UnsafeCell<BlockRng64<SharedBufferRng<WORDS_PER_SEED, SEEDS_CAPACITY, SourceType>>>>);

impl <const WORDS_PER_SEED: usize, const SEEDS_CAPACITY: usize, SourceType>
ThreadLocalSeeder<WORDS_PER_SEED, SEEDS_CAPACITY, SourceType> {
    fn new(source: SharedBufferRng<WORDS_PER_SEED, SEEDS_CAPACITY, SourceType>) -> Self {
        ThreadLocalSeeder(Rc::new(UnsafeCell::new(BlockRng64::new(source))))
    }

    fn get_mut(&self) -> &mut BlockRng64<SharedBufferRng<WORDS_PER_SEED, SEEDS_CAPACITY, SourceType>> {
        // SAFETY: Same as impl RngCore for ThreadRng: https://rust-random.github.io/rand/src/rand/rngs/thread.rs.html
        unsafe {
            self.0.get().as_mut().unwrap()
        }
    }
}

thread_local! {
    static DEFAULT_FOR_THREAD: ThreadLocalSeeder<8, 16, OsRng>
        = ThreadLocalSeeder::new(DEFAULT_ROOT.get_or_init(|| SharedBufferRngStd::new(OsRng::default())).clone());
}

impl <const WORDS_PER_SEED: usize, const SEEDS_CAPACITY: usize, SourceType>
RngCore for ThreadLocalSeeder<WORDS_PER_SEED, SEEDS_CAPACITY, SourceType> {
    fn next_u32(&mut self) -> u32 {
        self.get_mut().next_u32()
    }

    fn next_u64(&mut self) -> u64 {
        self.get_mut().next_u64()
    }

    fn fill_bytes(&mut self, dest: &mut [u8]) {
        self.get_mut().fill_bytes(dest)
    }

    fn try_fill_bytes(&mut self, dest: &mut [u8]) -> Result<(), Error> {
        self.get_mut().try_fill_bytes(dest)
    }
}

pub fn thread_seeder() -> ThreadLocalSeeder<8, 16, OsRng> {
    DEFAULT_FOR_THREAD.with(ThreadLocalSeeder::clone)
}

pub fn thread_rng() -> ReseedingRng<ChaCha12Core, ThreadLocalSeeder<8, 16, OsRng>> {
    let mut reseeder = thread_seeder();
    let mut seed = <ChaCha12Core as SeedableRng>::Seed::default();
    reseeder.fill_bytes(&mut seed);
    ReseedingRng::new(ChaCha12Core::from_seed(seed), 1 << 16, reseeder)
}

This is not future-proof against any change to the default algorithm, the reseeding threshold, or the use of Rc<UnsafeCell<_>>. It would be better if there was a method that would construct a ThreadRng given reseeder as the only parameter, and otherwise promise to have the same implementation details as thread_rng().

[dhardy]

Is there an actual blocking/performance issue? If so, you haven't outlined it clearly, merely stated that there is a theoretical one.

Reseeding on Linux uses getrandom which uses the getrandom system call which is non-blocking except in the case that the system RNG has not yet been initialized since boot. So I don't think there is an issue.

But anyway, for MC models you might want to use a local RNG like ChaCha8Rng with fixed seed just to get reproducible output?

[Pr0methean]

A little micro-benchmarking of https://github.com/Pr0methean/rng_buffer/blob/master/src/lib.rs (a wrapper that combines several consecutive getrandom calls per thread into one) shows that it improves performance just by amortizing the overhead of a system call. Sharing a buffer between threads, OTOH, makes the buffer access no faster than the syscall.

[dhardy]

That's a lot of code to batch calls to the OsRng...

... but what was the point of reseeding again?

1. To avoid wrapping ChaCha's counter at 1 << 68 words (2^70 bytes). But this number is huge, and this was never the goal of reseeding.
2. To recover unpredictability in a long-running server program in case (a) the OS originally yielded poor-quality random data (shouldn't happen) or program state was leaked.

So, if you're going to batch 16 new seeds, you might just as well set the reseeding interval to 16 times the original length instead, just so long as you don't exceed 2^70 bytes (impossible given that this is a 64-bit counter).