search

turboderp / exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs
MIT License
3.52k stars 271 forks source link

module 'exllamav2_ext' has no attribute 'count_match' #481

Closed abpani closed 3 months ago

abpani commented 4 months ago

while trying generator.warmup() shows me below error. torch 2.3 flash_attn 2.5.9

AttributeError: module 'exllamav2_ext' has no attribute 'count_match'

turboderp commented 4 months ago

You have an old version of exllamav2 installed with a prebuilt wheel while using a newer version of the repo.