search

LDLINGLINGLING / adan_application

个人项目地址，一些大语言模型和多模态模型的应用
119 stars 7 forks source link

请问MiniCPMV2_6_awq量化脚本可以做8bit量化吗 #2

Closed KevinSpringer1 closed 2 months ago

LDLINGLINGLING commented 3 months ago

应该可以，4bit和8bit就是改个参数而已，MiniCPMV2_6_awq/quantize.py这里修改w_bit为8就可以了