tracel-ai / burn

Burn is a new comprehensive dynamic Deep Learning Framework built using Rust with extreme flexibility, compute efficiency and portability as its primary goals.
https://burn.dev
Apache License 2.0
9k stars 445 forks source link

PRNG Seeding Guidance #2312

Open lancelet opened 1 month ago

lancelet commented 1 month ago

Request: can the documentation please provide guidance on setting the PRNG seed for consistent tensor initialization?

Running this code produces a different value on every run:

let device = NdArrayDevice::default();
let conv2d = Conv2dConfig::new([1, 1], [1, 1]).init::<NdArray>(&device);
dbg!(conv2d.weight.val().into_data().to_vec::<f32>().unwrap());

In many situations, having a different seed for each run is useful because it allows training from a different initialization. However, it is also frequently useful to fix the seed. For example, in testing scenarios, there are sometimes floating-point comparisons that succeed for particular seeds, but fail in a borderline way for others. To avoid spurious CI failures in such scenarios, fixing the seed to a "known good value for this test run, on my chosen CI backend" is an extremely useful bandaid.

laggui commented 1 month ago

Using the Backend::seed(seed) method should do the trick.

type B = NdArray;
let device = NdArrayDevice::default();
B::seed(42);
let conv2d = Conv2dConfig::new([1, 1], [1, 1]).init::<B>(&device);
dbg!(conv2d.weight.val().into_data().to_vec::<f32>().unwrap());

It's used across multiple unit tests and examples, but perhaps not documented enough 🤔 Perhaps a bit more info in the Backend section would help?