search

Adamliu1 / SNLP_GCW

3 stars 0 forks source link

Improve Reproducability according to pytorch guidelines #5

Open Willmish opened 6 months ago

Willmish commented 6 months ago

Guidelines: https://pytorch.org/docs/stable/notes/randomness.html

Inspiration from ADL GCW : https://github.com/Foalfloater66/c197-gcw/pull/20

Might need to set: https://pytorch.org/docs/stable/generated/torch.use_deterministic_algorithms.html#torch.use_deterministic_algorithms

Willmish commented 6 months ago

, Add quantisation flag!

Adamliu1 commented 2 months ago

NOTE and UPDATE on this issue. Since we are now using multiple-GPU, we may need to use torch.cuda.manual_seed_all according to https://pytorch.org/docs/stable/generated/torch.cuda.manual_seed_all.html and https://pytorch.org/docs/stable/generated/torch.cuda.manual_seed.html#torch.cuda.manual_seed

Willmish commented 1 month ago

well @Willmish will revisit