Docs: Add caveat about madsim reproducibility

[kwannoel]

Initially observed here: https://github.com/risingwavelabs/risingwave/issues/7901

Across various build environments the behaviour of RNG in a system may change, if they use from_entropy for instance. Or even just compilation might produce different ordering of rng writes and reads.

It should behave the same only with the same build. For example: we can run the original binary, it should yield the same result with the same SEED. Alternatively: Ensure build environment is the same (compiler, os, commit).

Should add some caveat about this in docs + output on how to reproduce when madsim cluster crashes.

[kwannoel]

may /assign me 😆

[wangrunji0408]

Sure, it would be great to have such documents for developers. Please feel free to add them. Thanks in advance!

[kwannoel]

More specific details:

• OS must be same.
• Commit must be same.
• Compiler toolchain must be same.

We can list these as some examples. I'm not sure if it's enough, maybe we can write some tests to verify 🤔