Open tzuhanchang opened 4 months ago
Add torch.nn.AdamW optimizer.
torch.nn.AdamW
Add torch.optim.AdamW and torch.optim.SGD. Also, add weight_decay and momentum options, but set their default values to 0.
torch.optim.AdamW
torch.optim.SGD
weight_decay
momentum
Add
torch.nn.AdamW
optimizer.