请问这个int4量化是怎么做的了，有相关的参考文件吗。

OpenBMB / MiniCPM-V

MiniCPM-V 2.6: A GPT-4V Level MLLM for Single Image, Multi Image and Video on Your Phone

Apache License 2.0

11.82k stars 829 forks source link

请问这个int4量化是怎么做的了，有相关的参考文件吗。 #139

Closed Alxemade closed 3 months ago

Alxemade commented 3 months ago

起始日期 | Start Date

No response

实现PR | Implementation PR

No response

摘要 | Summary

无

基本示例 | Basic Example

无

缺陷 | Drawbacks

无

未解决问题 | Unresolved questions

No response

Cuiunbo commented 3 months ago

Question：May I ask how this int4 quantization was done, is there a reference document for it? @tc-mb

tc-mb commented 3 months ago

起始日期 | Start Date

No response

实现PR | Implementation PR

No response

相关Issues | Reference Issues

No response

摘要 | Summary

无

基本示例 | Basic Example

无

缺陷 | Drawbacks

无

未解决问题 | Unresolved questions

No response

I'll give a int4/int8 introduction later this week.

liHai001 commented 2 months ago

请问下，有发 MiniCPM-Llama3-V-2_5 量化文档了不，比如GPTQ/Awq 4bit量化，没看到有文档 @tc-mb

tc-mb commented 2 months ago

请问下，有发 MiniCPM-Llama3-V-2_5 量化文档了不，比如GPTQ/Awq 4bit量化，没看到有文档 @tc-mb

人力不济，还没时间写。