Research purpose implementation of Tensor Train and Tensor Ring algorithms in Pytorch

Paper

System requirements

• Python 3
• CPU or NVIDIA GPU + CUDA

Dependencies and Getting Started

• torch >= 1.0.0
• torchvision >= 0.2.1
• numpy
• sympy
• scipy

You may install PyTorch using any suggested method for your environment here.

Also, after cloning the repo, you can run python setup.py install in the command line to install the required packages.

Setting up experiments

To check the experiments settings, see a file experiments.sh.

For example, to run an experiment for TT embedding layer you can run:

python train.py --embed_dim 256 --dataset imdb --embedding tt \
    --n_epochs 100 --d 3 --ranks 16 --gpu 1

Repository structure

The directory t3nsor contains classes and function for TT and TR decompositions, embedding layers and so on. The directory sentiment contains the models and experiment setting files.

Authors

• Valentin Khrulkov
• Oleksii Hrinchuk
• Leyla Mirvakhabova
• Elena Orlova
• Ivan Oseledets

If you use these algorithms in your research we kindly ask you to cite our work

@article{khrulkov2019tensorized,
  title={Tensorized {E}mbedding {L}ayers {F}or {E}fficient {M}odel {C}ompression},
  author={Khrulkov, Valentin and Hrinchuk, Oleksii and Mirvakhabova, Leyla and Orlova, Elena and Oseledets, Ivan},
  journal={arXiv preprint arXiv:1901.10787},
  year={2019}
}