-
The following simple script reproduces the issue:
model = torch.nn.Linear(1, 1)
optimizer = mechanize(torch.optim.AdamW)(model.parameters(), lr=1e-5)
x = torch.ones([5, 1])
out =…
-
-
The Adam optimizer can consume a large amount of GPU memory, potentially causing OOM (Out Of Memory) errors during training. To free up memory during forward/backward passes, there is a need for a fea…
-
If we enable expert parallelism, there will be two optimizers for dense parameters and expert parameters. When we call `optimizer.step() ` the two optimizers perform grad-norm for their own parameters…
-
Hello,
I am trying to use the DipEncoder class for a project. It's a very interesting model for the problem I am trying to solve as it allows for intrinsic optimization of the number of clusters. …
-
## Describe the bug
I am trying to use the Optimizer from clearml
When I launch the optimizer based on template task id, it is not picking up the hyperparameters arguments.
For clarification, my…
-
```bash
# Create your project directory then go into it:
mkdir -p ~/Sites/magento
cd $_
# Download the Docker Compose template:
curl -s https://raw.githubusercontent.com/markshust/docker-magent…
-
here is my code:
from torch_ema import ExponentialMovingAverage
model = ...
optimizer = ...
scheduler = ...
ema_model = ExponentialMovingAv…
-
### 🐛 Describe the bug
I was trying to get some toy training loop going and there were a few sharp bits that we should probably document in 1 place
1. Need to instantiate both `optimizer` and `c…
-
Current implementation is not friendly for the development of line search and will need some refactoring.