search

CuriousAI / mean-teacher

A state-of-the-art semi-supervised method for image recognition
https://arxiv.org/abs/1703.01780
Other
1.56k stars 331 forks source link

what is the difference between input and ema_input #17

Open marcosly opened 5 years ago

marcosly commented 5 years ago

I have no idea why have 2 input in train_loader

tarvaina commented 5 years ago

The algorithm works by having two models that take a similar input and produce a similar output: the student model, which is a normal convolutional neural network, and a teacher model, which is the same as student except its weights are exponential moving average (EMA) of the student network. Before feeding the input to the networks, the algorithm adds noise to the input. This is done for the networks separately, sampling noise twice from the same distribution.

In the Pytorch code, the sampling of noise twice is handled by TransformTwice class (for example here). The resulting transformation takes one input image and returns two noisy versions of it. The transformation is wrapped in to the train_loader, and when the train_loader is iterated (here), it returns those two noisy input images: input and ema_input. The ema_ prefix there just means that it is the one to be fed to the ema_model, i.e. the teacher network.

marcosly commented 5 years ago

oh, I get it ,thanks for your work!

ZhuMengliang commented 3 years ago

The algorithm works by having two models that take a similar input and produce a similar output: the student model, which is a normal convolutional neural network, and a teacher model, which is the same as student except its weights are exponential moving average (EMA) of the student network. Before feeding the input to the networks, the algorithm adds noise to the input. This is done for the networks separately, sampling noise twice from the same distribution.

In the Pytorch code, the sampling of noise twice is handled by TransformTwice class (for example here). The resulting transformation takes one input image and returns two noisy versions of it. The transformation is wrapped in to the train_loader, and when the train_loader is iterated (here), it returns those two noisy input images: input and ema_input. The ema_ prefix there just means that it is the one to be fed to the ema_model, i.e. the teacher network.

Thank you for your great answer!