search

the-crypt-keeper / can-ai-code

Self-evaluating interview for AI coders
https://huggingface.co/spaces/mike-ravkine/can-ai-code-results
MIT License
513 stars 29 forks source link

Evaluate ikawrakow/mixtral-8x7b-quantized-gguf (improved mixtral GGUF) #144

Closed the-crypt-keeper closed 8 months ago

the-crypt-keeper commented 8 months ago

Big improvements to Q2 and Q3 claimed while remaining compatible with GGUF

the-crypt-keeper commented 8 months ago

Also interesting and freshly merged: https://huggingface.co/ikawrakow/various-2bit-sota-gguf

the-crypt-keeper commented 8 months ago

Complete.

The new q3k-medium appears to be broken.

q3k-small and q2k outperform TheBloke's GGUF quants.

The new 2.1bpw sota encoding does well at junior-v2 but rather poorly on senior