search

jonbarron / robust_loss_pytorch

A pytorch port of google-research/google-research/robust_loss/
Apache License 2.0
656 stars 88 forks source link

About the dimensionality of the residual matrix #9

Closed ahmed-fau closed 4 years ago

ahmed-fau commented 5 years ago

Hi,

Thanks for sharing this nice work. I am trying to use the adaptive loss for regularizing my conditional GAN model instead of using the L1 norm. However, I cannot get reliable performance.

I think the description of the dimensionality of the residual input x is a bit unclear. What I have understood is: the input to the function lossfun should be a 2d matrix with number of rows equals the batch size and number of columns equals the flattened feature map, so that if I have a tensor of dimensionality [batch,channel,squence or map] then I should reshape it to be [batch, channels * sequence or map] ... is this correct?

If so, does this mean that the member variable self.num_dims should also equal channels * sequence or map?

Many thanks in advance

jonbarron commented 5 years ago

Yes, the input to general.lossfun() adaptive.lossfun() must be a 2d matrix, of size [num_batches, num_dims]. It sounds like in your case, you should reshape to "channels * sequence or map" and set self.num_dims to that as well.