DRAGNLabs / 301r_retnet

2 stars 1 forks source link

Random Seed #13

Closed nprisbrey closed 6 months ago

nprisbrey commented 6 months ago

Create the command line argument --rand-seed to allow the user to set the random seed in PyTorch. This is beneficial because it will facilitate better comparisons between different model types, decreasing the variance error of the model.

In the future, we could look into implementing torch.use_deterministic_algorithms(True) as described in the PyTorch Reproducibility Documentation. For me, however, this causes CUDA errors that would limit its performance, making me tend towards not implementing this functionality unless deemed necessary in the future.