fastai / course-v3

The 3rd edition of course.fast.ai
https://course.fast.ai/
Apache License 2.0
4.9k stars 3.55k forks source link

whether using "self.opt" or "self.run.opt" #541

Open hnczwj2008 opened 4 years ago

hnczwj2008 commented 4 years ago

Hi, I'm reading the course-v3 ***.ipynb , https://github.com/fastai/course-v3/blob/master/nbs/dl2/09_optimizers.ipynb , and I was wondering whether should I use "self.opt" or "self.run.opt" in the callbackhanders. such as: "class Recorder(Callback):" ,"class ParamScheduler(Callback):" and so on. here is the code I am geting confused:

class Recorder(Callback):
    def begin_fit(self): self.lrs,self.losses = [],[]

    def after_batch(self):
        if not self.in_train: return #or using: if not self.run.in_train: return
        self.lrs.append(self.opt.hypers[-1]['lr'])  #or using:  self.lrs.append(self.run.opt.hypers[-1]['lr']) 
        self.losses.append(self.loss.detach().cpu()) #or using:  self.losses.append(self.run.loss.detach().cpu()) 

class ParamScheduler(Callback):
    _order=1
    def __init__(self, pname, sched_funcs):
        self.pname,self.sched_funcs = pname,listify(sched_funcs)

    def begin_batch(self): 
        if not self.in_train: return
        fs = self.sched_funcs
        if len(fs)==1: fs = fs*len(self.opt.param_groups) #or using:  if len(fs)==1: fs = fs*len(self.run.opt.param_groups)
        pos = self.n_epochs/self.epochs #or using: pos = self.run.n_epochs/self.run.epochs
        for f,h in zip(fs,self.opt.hypers): h[self.pname] = f(pos) #or using:   for f,h in zip(fs,self.run.opt.hypers): h[self.pname] = f(pos)

Because , I find the attributes: "opt , loss,n_epochs, epochs.... " are only defined in the Learner(), they are passed to every CallbackHander by the Callback() class through the code:

class Callback():
    _order=0 
    def set_runner(self, run): self.run=run 
prats0599 commented 4 years ago

Both self.opt and self.run.opt refer to the same object(i.e. the optimizer in the Learner class which gets passed to Runner).