Closed varunsatish closed 2 months ago
Previously, we were only able to do evaluation at the end of each epoch. This was not ideal for us given that our experiments were going to begin using just one epoch.
I amended the code (with a lot of Claude-ing) to do evaluation at intervals determined by the logging_steps
flag. For example, we can choose to do evaluation every 10, 1000, 1000 steps.
We can still do evaluation at each epoch by determining the number of steps in each epoch. This is determined by the quantity: # training sample / batch size.
CC: @msalganik
👍
Llama-recipes supports evaluation at each epoch, but not the number of steps natively. It would be useful for us to add evaluation by number of steps.