Add more initialization methods

[merajhashemi]

Hello. Currently, the default initialization for the linear layers is a random tensor with std=0.02. I suggest adding more initialization methods such as Xavier (Glorot) and Kaiming (He). http://proceedings.mlr.press/v9/glorot10a/glorot10a.pdf https://arxiv.org/pdf/1502.01852.pdf I have also previously experienced that initializing the cores so that the reconstructed matrix has expected Frobenius norm of 1 is sometimes helpful. Not sure if it's useful to be added though.

[merajhashemi]

I guess with the new release (Thanks @JeanKossaifi 😄) there is no need for new initializations as PyTorch’s gaussian Xavier and Kaiming initializations will work by default.

Example:

import tltorch
import torch

tt_tensor = tltorch.TensorizedMatrix.new((4, 4),(3, 3), rank='same', factorization='TT')
torch.nn.init.kaiming_normal_(tt_tensor)
torch.nn.init.xavier_normal_(tt_tensor)

Also I think the init module is currently redundant—as it’s only a duplicate of the TensorizedMatrix.normal_ methods—and could be removed.