ddkang / loss_dropper

Apache License 2.0
51 stars 9 forks source link

hi #5

Open zmxsss opened 3 years ago

zmxsss commented 3 years ago

If my loss is one-dimensional, do I need to become two-dimensional? use loss = loss.view(-1, batch_size)

zmxsss commented 3 years ago

my loss.shape [1280.]

ddkang commented 3 years ago

With our implementation of loss dropping, we require a batch dimension. Doing that should work.

It should be a simple code change to deal with one-dimensional losses though!

zmxsss commented 3 years ago

my code: loss = self.loss_fn(tgt_word_logprobs, tgt_word_labels) loss=loss.view(-1,bs) loss = loss.mean(dim=0) aa = self.dropper(loss) loss *= aa loss = loss.mean() loss = loss.view(-1) when run 6 epoch , error: RuntimeError: shape '[-1, 64]' is invalid for input of size 120, do you know why?

ddkang commented 3 years ago

I don't think you need the mean if it's already one dimensional.