-
I tired to follow the example of using MAGRAD optimizer
```
import torch_optimizer as optim
optimizer = optim.MAGRAD(model.parameters(), lr=0.1)
optimizer.zero_grad()
loss_fn(model(in…
-
huggingface train의 optimizers인자로 (optmizer, lr_shceduler)를 주면 해당 값을 train시 이용하게 됩니다.
MADGRAD는 상민님이 추천해준 optimizer로 2021년1월26일에 처음 소개된 방식입니다.
해당 방식을 이용하기 위해서 facebook research의 madgrad.py 코드를 이용하…
-
https://github.com/facebookresearch/madgrad
It looks better than SGD and ADAM. I wonder if it can be implemented to DFL
-
Hello, should I use what kind of hyper-paramter for the first try? For example, learning rate, AdamW or Madgrad?
-
Hi,
I've been setting up with swin_transformer but having a hard time getting it to actually train.
I figured one immediate issue is the lack of init, so I'm using the truncated init setup from rw…
-
Calling Ranger21 with mostly default parameters:
```
optimizer = ranger21.Ranger21(
net.parameters(), lr=0.001, num_epochs=50, weight_decay=1e-5,
num_batches_per_epoch=len(tr…
-
Are there any recommended settings for Transformer Language modeling?
-
Hi all,
I'm trying to use sjSDM to run joint species distribution model. But when I run the model it reminds:
Error: PyTorch not installed
I have tried all methods …
-
Hi @lessw2020, thanks for the very nice work!
I noticed that in this Ranger21, the optimizer is tightly coupled with the lr scheduler, could you guide me how I can decouple them?
hiyyg updated
2 years ago
-
alstro is reporting increased diversity when doing that
example script that needs
* https://github.com/crowsonkb/deep-image-prior
* this needs to be replaced https://github.com/crowsonkb/v-dif…