search

titu1994 / keras-one-cycle

Implementation of One-Cycle Learning rate policy (adapted from Fast.ai lib)
MIT License
285 stars 79 forks source link

LRFinder smoothing #2

Open yaroslavb1 opened 6 years ago

yaroslavb1 commented 6 years ago

The LR smoothing in line 336 isn't done right. running_loss = self.loss_smoothing_beta loss + (1. - self.loss_smoothing_beta) loss The first loss should be self.runningloss, and self.runningloss should be updated right after.

tyokota commented 5 years ago

@yaroslavb1 could you please provide code for what you mean here?

yaroslavb1 commented 5 years ago

@tyokota 

# smooth the loss value and bias correct
running_loss = self.loss_smoothing_beta * self.running_loss_ + (1. - self.loss_smoothing_beta) * loss
self.running_loss_ = running_loss 
running_loss = running_loss / (1. - self.loss_smoothing_beta ** self.current_batch_)