clovaai / bros

Apache License 2.0
155 stars 23 forks source link

Clarification regarding `num_samples_per_epoch` #16

Closed suyogdahal closed 1 year ago

suyogdahal commented 2 years ago

Could you guys please clarify whether num_samples_per_epoch in the config files refers to the total number of documents in the training set or does it mean something else?

I set the num_samples_per_epoch to the number of docs in my training set, however, the LRScheduler warmup is not working as expected.

logan-markewich commented 1 year ago

@suyogdahal num_samples_per_epoch is referring to the dataset size AFTER batching. So if you have a batch size of 4, divide the size of your training set by 4 to get the appropriate schedule