-
* ~~SOAP~~
* Muon (add schedules)
* Shampoo (only DistributedShampoo)
* ~~Adam-mini~~
* ~~Lion~~
* ~~Sophia~~
* ~~AdEMAMix~~
* ~~Schedule-Free~~
* Adafactor
* Adalayer
* ~~Signum, signSGD~~
…
-
-
Hi, I'm trying to use adahessian in TensorFlow for a simple regression experiment but having trouble.
I have a simple example in this google colab notebook: https://colab.research.google.com/drive/…
-
Hi
I am using your library and appreciate all the work you have put into this capability. I started using the adahessian optimizer and found that my GPU memory would increase until it used all my GP…
-
torch_optimizer/adahessian.py", line 128, in get_trace
hutchinson_trace.append(tmp_output)
UnboundLocalError: local variable 'tmp_output' referenced before assignment
-
Hi, yiliu, I notice that you have set the federated unlearning repo private or deleted it, could you please share it again? I am interested in your adahessian optimizer to accelerate local training.
-
Hi,
Thanks for this great work. Recently, we tried to train ResNext-50 on ImageNet classification using AdaHessian. The implementation we used is from https://github.com/davda54/ada-hessian.
How…
-
hi, I find some err when trying to use the adahessian optimizer, like
''AttributeError: 'torch._C.Generator' object has no attribute 'device'",
pytorch version: 1.1.0
-
Hello! Our static bug checker has found a performance issue in adahessian_tf/run_experiments.py and adahessian_tf/cifar_training_tools.py: [`cifar_training`](https://github.com/amirgholami/adahessian…
-
Hi,
I am new to this ecosystem and was exposed to it via the talk by Dhairya at JuliaCon2020. I thought to ask if there are any plans for you to include 2nd order Optimizers, that were recently intro…