-
Hey, great work with lingua ! I was wondering if you'd consider adding support for the distributed shampoo optimizer.
-
Hey,
I have been using Meta's Implementation of Distributed Shampoo and am seeing ~20% faster convergence of transformer based models compared to AdamW. [Simo Ryu](https://x.com/cloneofsimo/status/…
-
### What
- We currently support urls like /[tag type]/[tag value] to display products that have a specific tag.
- e.g. all Shampoos: https://world.openbeautyfacts.org/category/shampoos
- This fea…
-
I would like to fix the bug and more data to shopping cart and can a new category as make over in which the body care like shampoo ,lotions can come under
-
* ~~SOAP~~
* Muon (**FIX IT**)
* Shampoo (basic)
* ~~Adam-mini~~
* ~~Lion~~
* Sophia
* ~~AdEMAMix~~
* ~~Schedule-Free~~
* Adafactor
* Adalayer
* ~~Signum, signSGD~~
* AdaHessian
* Prodigy
…
-
Using the latest main to train a YoloV9e object detector:
```
[rank0]: train_one_epoch(train_loader, model, args, model_dtype)
[rank0]: File "/mnt/dingus_drive/catid/train_detector/train.py…
-
I am trying to reimplement u-MuP in JAX and using this repo as a reference. I cannot figure out why Adam and SGD use different lr scaling factors: $\frac{1}{\sqrt{\text{fan-in}}}$ for [Adam](https://g…
-
I run shampoo.test() and get an error. Could someone help me fix this problem?
It says **TypeError: 'float' object cannot be interpreted as an index**
![image](https://user-images.githubusercontent.…
-
The conformer workload hangs when run with shampoo training algorithm.
## Description
Traceback
```
I0505 23:26:00.158526 139795269302080 submission_runner.py:319] Starting training loop.
I0505…
-
Load directly to production database, after #146 is complete