mixup doesn't work - self.in_train not found

[rmkn85]

Tried to use Mixup, like in the notebook: https://github.com/fastai/fastai_dev/blob/master/dev/19_callback_mixup.ipynb

mixup = MixUp(0.5)
learn = Learner([...], cbs=mixup)

But getting self.in_train (which is referred in MixUp.lf()) not found. Indeed I didn't find it anywhere in the code.

[dcato98]

I submitted a PR to fix this. In the meantime, try this version of MixUp:

class MixUp(Callback):
    run_after=[Normalize, Cuda]
    def __init__(self, alpha=0.4): self.distrib = Beta(tensor(alpha), tensor(alpha))
    def begin_fit(self): self.old_lf,self.learn.loss_func = self.learn.loss_func,self.lf
    def after_fit(self): self.learn.loss_func = self.old_lf

    def begin_batch(self):
        if not self.training: return
        lam = self.distrib.sample((self.y.size(0),)).squeeze().to(self.x.device)
        lam = torch.stack([lam, 1-lam], 1)
        self.lam = lam.max(1)[0]
        shuffle = torch.randperm(self.y.size(0)).to(self.x.device)
        xb1,self.yb1 = tuple(L(self.xb).itemgot(shuffle)),tuple(L(self.yb).itemgot(shuffle))
        nx_dims = len(self.x.size())
        self.learn.xb = tuple(L(xb1,self.xb).map_zip(torch.lerp,weight=unsqueeze(self.lam, n=nx_dims-1)))

    def lf(self, pred, *yb):
        if not self.training: return self.old_lf(pred, *yb)
        with NoneReduce(self.old_lf) as lf:
            loss = torch.lerp(lf(pred,*self.yb1), lf(pred,*yb), self.lam)
        return reduce_loss(loss, getattr(self.old_lf, 'reduction', 'mean'))

[rmkn85]

nx_dims assignment is commented out, and then used in next line?

[dcato98]

Oops, good catch. I've updated the code in my previous comment.

[sgugger]

Pr merged, so this should be fixed. Thanks!