search

the-crypt-keeper / can-ai-code

Self-evaluating interview for AI coders
https://huggingface.co/spaces/mike-ravkine/can-ai-code-results
MIT License
537 stars 30 forks source link

Evaluate llama2 with low-bit-llama #99

Closed the-crypt-keeper closed 4 months ago

the-crypt-keeper commented 1 year ago

https://github.com/GreenBitAI/low_bit_llama

2 bit quants with performance figures that are difficult to believe.

Model: https://huggingface.co/GreenBitAI/LLaMA-2-70B-2bit-groupsize8

the-crypt-keeper commented 10 months ago

the quip-sharp method #122 is newer and has better 2-bit perplexity, going to leave this open but focus 2-bit efforts there for now

the-crypt-keeper commented 4 months ago

Closing out all old 2-bit quants.