Open odelalleau opened 1 month ago
Describe the bug
NB: this was first noticed by @eloialonso
Code related to train_valid_test_num_samples seems weird -- at least for RM training (I haven't checked other algos):
train_valid_test_num_samples
[-1 * cfg.model.global_batch_size] * 3
=> probably that some code clean-up is needed (?)
Describe the bug
NB: this was first noticed by @eloialonso
Code related to
train_valid_test_num_samples
seems weird -- at least for RM training (I haven't checked other algos):[-1 * cfg.model.global_batch_size] * 3
is meant to do (link)=> probably that some code clean-up is needed (?)