-
conversion problem in Appendix D probably due to malformed:ltx:listing.
>
> Appendix D Implementation details
> D.1 Creating a MomentumNet with a MLP
>
> \parimport torch
> import torch.nn…
-
Thanks for your interesting work!
The [Reformer](https://arxiv.org/abs/2001.04451) uses RevNet in a clever way. They double the dimension of `x` such that for `x1,x2=split(x)` both `x1` and `x2` ha…
-
This issue may or may not contain my notes about implementing and training reversble models.
_From correspondence with @TimDettmers, @yhn112 , @borzunov, @mryab_
__Why?__ Reversible models are one…
-
These are notes from our discussion w/ @pierreablin on the design of the benchmark for NN. Feel free to comment/add/edit stuff.
### Critical steps for CIFAR10 training
There are a few critical s…
-
Inplace BatchNorm seems to be developed by Mapillary here: https://github.com/mapillary/inplace_abn
This would be a very nice addition to core PyTorch (for memory savings).
cc @ezyang @gchanan @…
-
- https://arxiv.org/abs/2102.07870v2
- 2021
バックプロパゲーションを用いた深い残差ニューラルネットワーク(ResNets)の学習には、ネットワークの深さに応じて線形に増加するメモリコストがかかります。
この問題を回避する方法として、可逆的なアーキテクチャを使用することが挙げられる。
本論文では、モーメンタム項を追加することで、ResNetの順…
e4exp updated
3 years ago
-
I tested the memory consumption of resnet 101 and the corresponding momentum version using the scripts below. However, difference in memory consumption between these 2 models is only a few hundreds MB…
-
In this example setting gamma=0.0 lead to mnet1 and net having two different outputs.
However, everything is fine with resnet18.
(it seems that it apply the residual connection of ResBlock after…