Open JoyBoy-Su opened 1 month ago
Can I work on it? and we need to use dynamic quantization of pytorch right?
Of course! Thanks a lot! You could submit a PR!
PR created #21
What's the status of this? want to use quantized model on Windows
I am currently adding a feature so that you can use -q and you'll be able to use a quantised model for inference
Speeding up Inference with Quantized models