search

facebookresearch / fairseq2

FAIR Sequence Modeling Toolkit 2
https://facebookresearch.github.io/fairseq2/
MIT License
678 stars 78 forks source link

Introduce temporary_manual_seed() #591

Closed cbalioglu closed 3 months ago

cbalioglu commented 3 months ago

This PR introduces a new temporary_manual_seed() function which is a "cleaner" way to torch.random.fork_rng() + torch.manual_seed() as it internally uses RngBag to correctly set the seed only for the passed devices. It also extends reset_parameters() and to_device() utility functions with an extra convenience seed parameter.

shagunsodhani commented 3 months ago

This seems cool! Could you share an example of where this is commonly used ?

cbalioglu commented 3 months ago

We use it in reset_parameters() and to_device() functions in fairseq2.nn.utils.module.

We also use it in unit/integration tests to avoid changing the global seed.