search

ermongroup / ssdkl

Code that accompanies the paper Semi-supervised Deep Kernel Learning: Regression with Unlabeled Data by Minimizing Predictive Variance
74 stars 29 forks source link

Batch setting in Mean-teacher optimization #2

Closed toushi68 closed 5 years ago

toushi68 commented 5 years ago

Hi, I have a question about the batch setting inside the optimization loop for mean teacher. Please have a look of the following codes for what I am talking about. https://github.com/ermongroup/ssdkl/blob/eba7023179597316f57bb536d8fb1562855ec981/labelprop_and_meanteacher/mean_teacher.py#L152 I am trying to understand why the call "create_feed_dicts" is inside the optimization loop. In each optimization cycle train_fd gives the same (entire) training set only once without any "batch" information, the only variations could be the teacher and student noise. Did I miss any thing? Thanks!

sangmichaelxie commented 5 years ago

The other thing that changes is the unlabeled data is resampled by reshuffling. There is some unnecessary reshuffling of labeled data, but we the entire training set at every batch so this doesn't affect anything.

toushi68 commented 5 years ago

So it's save to move the call (of create_feed_dicts) out of the optimization loop? Because I want to use the "mini-batch" to speed up when the training set is getting bigger.

sangmichaelxie commented 5 years ago

We used the entire labeled set in the minibatch to mimic the training of SSDKL, which requires the entire labeled set.

If you mean simply move the function call outside the loop, this won't work since there will be no resampling for unlabeled data. Also note that if you want to minibatch, you still need to create feed dicts with minibatches inside the optimization loop.

On Sun, Mar 10, 2019 at 6:07 AM toushi68 notifications@github.com wrote:

So it's save to move the call (of create_feed_dicts) out of the optimization loop? Because I want to use the "mini-batch" to speed up when the training set is getting bigger.

— You are receiving this because you commented.

Reply to this email directly, view it on GitHub https://github.com/ermongroup/ssdkl/issues/2#issuecomment-471294410, or mute the thread https://github.com/notifications/unsubscribe-auth/AHidSiVFzj8pwEF_qUVXZRo6S-gv0n0Eks5vVQOQgaJpZM4bmuF_ .

toushi68 commented 5 years ago

got it, thanks!