Factorized linear supports implementation switch and gradient checkpoint

tensorly / torch

TensorLy-Torch: Deep Tensor Learning with TensorLy and PyTorch

http://tensorly.org/torch/

BSD 3-Clause "New" or "Revised" License

74 stars 19 forks source link

Closed JeremieMelo closed 2 years ago

JeremieMelo commented 2 years ago

support implementation switches between factorized and reconstructed gradient checkpointing for memory-efficient training-mode forward function.

JeanKossaifi commented 2 years ago

Looks good, thanks @JeremieMelo, merging!