search

pytorch / FBGEMM

FB (Facebook) + GEMM (General Matrix-Matrix Multiplication) - https://code.fb.com/ml-applications/fbgemm/
Other
1.2k stars 498 forks source link

Why is there no implementation of adamw optimizer. Is there a plan for development? #2969

Closed maofagui closed 2 months ago

sryap commented 2 months ago

We don't have a plan to develop Adamw. However, your contribution is welcomed :)

JacoCheung commented 2 weeks ago

I was wondering shouldn't Adam in FBGEMM be exactly the AdamW? @sryap See codegen https://github.com/pytorch/FBGEMM/blob/v1.0.0/fbgemm_gpu/codegen/genscript/optimizers.py#L966-L993