Be ware of torch.FloatTensor(1). Should use torch.zeros(1)

[huangzhii]

Still for the orthogonal regularization piece of code orth_loss = Variable(torch.FloatTensor(1), requires_grad=True) Use torch.FloatTensor is dangerous as per soumith (https://github.com/pytorch/tutorials/issues/41).

Basically, torch.FloatTensor will create a Tensor with uninitialized memory instead of zero. The memory can contain any garbage, as it is uninitialized. This is intentional. If you want zeroed Tensors, you can use torch.zeros(1)

[kevinzakka]

I've finally updated the regularizations to use tensors rather than Variable and fixed this as well. Thanks @huangzhii !