During tests I noticed that the predicted time per epoch was off. When running on a g4dn.2xlarge the duration was estimated at 10 hours but it took roughly 12. Running on a g4dn.xlarge the estimate is still 10 hours but it takes roughly 24.
This PR fixes the prediction by including the time of the enumerate() function which was ignored in the previous timing logic.
Also it uses the entire number of iterations to sample which should improve the prediction further.
see [6571761]
Furthermore this PR also updates the GLINT360K loader to only log up to 100 statements and include a log time [6b3a7b6]
For more generality it uses the length of the loader instead of a constant while processing [9189e22]
During tests I noticed that the predicted time per epoch was off. When running on a g4dn.2xlarge the duration was estimated at 10 hours but it took roughly 12. Running on a g4dn.xlarge the estimate is still 10 hours but it takes roughly 24.
This PR fixes the prediction by including the time of the
enumerate()function which was ignored in the previous timing logic.Also it uses the entire number of iterations to sample which should improve the prediction further. see [6571761]
Furthermore this PR also updates the GLINT360K loader to only log up to 100 statements and include a log time [6b3a7b6]
For more generality it uses the length of the loader instead of a constant while processing [9189e22]