search

turboderp / exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs
MIT License
3.23k stars 238 forks source link

Loras: Remove qkv assertion #259

Closed bdashore3 closed 6 months ago

bdashore3 commented 6 months ago

QKV embeddings no longer exist in config, so this assertion will always fire due to config having QKV as None.