Can you explain to me what kind of decay function this is?
I've only seen linear, inverse of time and power, like the ones stated in http://www.ijmo.org/vol6/504-M08.pdf, formulas 5,6,7
def asymptotic_decay(learning_rate, t, max_iter):
"""Decay function of the learning process.
Parameters
learning_rate : float
current learning rate.
t : int
current iteration.
max_iter : int
maximum number of iterations for the training.
"""
return learning_rate / (1+t/(max_iter/2))
Hello! :)
Can you explain to me what kind of decay function this is? I've only seen linear, inverse of time and power, like the ones stated in http://www.ijmo.org/vol6/504-M08.pdf, formulas 5,6,7
def asymptotic_decay(learning_rate, t, max_iter): """Decay function of the learning process. Parameters