search

mit-han-lab / nunchaku

SVDQuant: Absorbing Outliers by Low-Rank Components for 4-Bit Diffusion Models
https://hanlab.mit.edu/projects/svdquant
Apache License 2.0
355 stars 17 forks source link

Is that possible to support any torch model? #30

Open microbenh opened 6 days ago

microbenh commented 6 days ago

Cool job. The quantification method mentioned in the article has good generality. Is there any possibility to directly support any torch model, without such a complex usage method. The ideal method is to take a torch model, be able to use Python code to quantize the model, save it, and then use it directly.

lmxyy commented 2 days ago

We are continuing improving our codebase and will modularize our engine. Stay tuned.