支持 4/8 bit 量化运行

Ucas-HaoranWei / Vary-toy

Official code implementation of Vary-toy (Small Language Model Meets with Reinforced Vision Vocabulary)

565 stars 41 forks source link

Open yazheng0307 opened 5 months ago

yazheng0307 commented 5 months ago

感谢作者团队的开源工作！请问Vary-toy的demo是否支持支持 4/8 bit 量化运行

Ucas-HaoranWei commented 5 months ago

线上demo是fp16运行的，目前还未支持4/8量化