Weird RNG behavior (running rust binary)

[dmitry-worker]

It seems that I have a weird random generator issue:

use rand::Rng;
use rand::SeedableRng;
use rand_chacha::ChaChaRng;

pub fn main() {
    let mut rng = ChaChaRng::seed_from_u64(0x42);
    for _ in 1..1000 {
        let x: [u8; 8] = rng.gen();
        println!("{:?}", x);
    }
}

After ~100 iterations (approximately 1 kilobyte of random data) it starts to generate the same values, repeating each 1, 2 or 4:

[143, 209, 221, 216, 248, 42, 173, 182]
[196, 71, 126, 101, 236, 239, 50, 206]
[143, 209, 221, 216, 248, 42, 173, 182]
[196, 71, 126, 101, 236, 239, 50, 206]
[197, 244, 234, 254, 111, 244, 53, 113]
[74, 199, 130, 9, 246, 198, 169, 107]
[197, 244, 234, 254, 111, 244, 53, 113]
[74, 199, 130, 9, 246, 198, 169, 107]
[169, 224, 50, 100, 60, 228, 4, 26]
[60, 29, 188, 159, 8, 15, 198, 36]
[169, 224, 50, 100, 60, 228, 4, 26]
[60, 29, 188, 159, 8, 15, 198, 36]
[169, 224, 50, 100, 60, 228, 4, 26]
[60, 29, 188, 159, 8, 15, 198, 36]
[169, 224, 50, 100, 60, 228, 4, 26]

I'm not sure it's a problem with rust / rng implementation or something like that. But qemu works as expected. Any hints for fixing that?

[jart]

That doesn't look good. Could you help us troubleshoot this?

[jart]

@dmitry-worker could you attach a static binary that reproduces this issue, to this issue tracker? Thanks!

[jart]

I'm unable to reproduce this. I modified your program slightly to output binary. The outputted bytes look consistent with strongly random data.

Then I tried cargo build --release and got this:

Looking at the blink -es strace log, this doesn't appear to be calling getrandom() and is generating the random data on its own. Therefore this is probably some kind of opcode bug.

[jart]

I've narrowed it down to one of the SSSE3 instructions. If SSSE3 is disabled in CPUID, it works.

[jart]

We have fuzz tests for every ssse3 instruction except for PALIGNR so I'm going to start there.

PSIGNB      fuzzed
PSIGNW      fuzzed
PSIGND      fuzzed
PABSB       fuzzed
PABSW       fuzzed
PABSD       fuzzed
PALIGNR
PSHUFB      fuzzed
PMULHRSW    fuzzed
PMADDUBSW   fuzzed
PHSUBW      fuzzed
PHSUBD      fuzzed
PHSUBSW     fuzzed
PHADDW      fuzzed
PHADDD      fuzzed
PHADDSW     fuzzed

[jart]

It looks like SHUFPD and SHUFPS were to blame. This was partially fixed in #68 but fuzz testing revealed there were more opportunities for improvement. I'm pushing a change presently which I've confirmed completely fixes this issue, and ensures it'll never regress again. Your Rust ChaCha generator under blink now produces perfectly consistent output with itself run outside blink. Enjoy! Thanks for the report.

[dmitry-worker]

I'm sorry that I couldn't participate in it during last week... had an overflow. :'-( But you found the source of the bug and fixed it - it's just amazing!

Thank you for the great job! <3