4bit or 8bit quantization?

salesforce / CodeT5

Home of CodeT5: Open Code LLMs for Code Understanding and Generation

https://arxiv.org/abs/2305.07922

BSD 3-Clause "New" or "Revised" License

2.74k stars 401 forks source link

Open PL2584718785 opened 1 year ago

PL2584718785 commented 1 year ago

《Salesforce/instructcodet5p-16b》 Are there any plans of releasing 4bit versions support for this? Thanks.