search

tensorly / torch

TensorLy-Torch: Deep Tensor Learning with TensorLy and PyTorch
http://tensorly.org/torch/
BSD 3-Clause "New" or "Revised" License
70 stars 18 forks source link

CP-FactorizedLinear Forward Propagation #14

Closed deep-deep-learning closed 2 years ago

deep-deep-learning commented 2 years ago

Hello,

I started working on factorizing the forward propagation of FactorizedLinear that currently reconstructs the matrix and use Linear.

I started with the JupyterNotebook to check the results and equations but will start implementing it directly on the FactorizedLinear class if you give me a go.

Thank you.

Best, Christian

deep-deep-learning commented 2 years ago

Hi Jean,

I totally forgot to introduce myself in the previous message. I am a Master's student in UC Santa Barbara working in the same lab as Cole Hawkins who contributed with the factorized embeddings layer. I am grateful to be able to work on the tensorly library.

You can check my result here: https://github.com/deep-deep-learning/torch/blob/main/notebooks/cp_factorized_forward.ipynb.

And, I just updated factorized forward propagation for factorization='cp', n_layer=1 case. Please let me know if you think that editing functionals.factorized_linear is better.

Thank you.

Best, Christian