search

the-crypt-keeper / can-ai-code

Self-evaluating interview for AI coders
https://huggingface.co/spaces/mike-ravkine/can-ai-code-results
MIT License
537 stars 30 forks source link

Support FLUTE quantization #217

Open the-crypt-keeper opened 3 months ago

the-crypt-keeper commented 3 months ago

https://github.com/HanGuo97/flute?tab=readme-ov-file

Llama3 and Gemma2 supported, this one claims to be very fast with vLLM.