search

huggingface / optimum-quanto

A pytorch quantization backend for optimum
Apache License 2.0
803 stars 58 forks source link

Add MSE Optimizer #135

Closed shuokay closed 6 months ago

shuokay commented 7 months ago

The goal of this issue is to implement MSE Optimizer. I have tested it on CNN models and Transformer-typed models(but not LLMs), mse worked pretty well. What's more, MSE is simple and intuitive.

dacorvo commented 7 months ago

Feel free to give it a try implementing your own Optimizer classes.

shuokay commented 7 months ago

@dacorvo I've put a draft here https://github.com/huggingface/quanto/pull/138. I don't have much free time during the weekdays, so I'm planning to finish it this weekend. 😂

dacorvo commented 7 months ago

Nice ! Please add a description to the pull-request, mentioning this issue.