Open Ayushk4 opened 1 year ago
Refer https://github.com/FMInference/FlexGen - they have explored storing cache at 4-bit quantization.
Refer https://github.com/FMInference/FlexGen - they have explored storing cache at 4-bit quantization.