When select downloaded model, it show error "DLL load failed while importing exllamav2_ext"

ParisNeo / lollms-webui

Lord of Large Language Models Web User Interface

https://parisneo.github.io/lollms-webui/

Apache License 2.0

4.07k stars 515 forks source link

When select downloaded model, it show error "DLL load failed while importing exllamav2_ext" #527

Open tomudo opened 2 months ago

tomudo commented 2 months ago

Fresh install v9.6. After install complete. When select model,it show error.

Expected Behavior

Can load model

Current Behavior

When select downloaded model, it show error "DLL load failed while importing exllamav2_ext"

Steps to Reproduce

Please provide detailed steps to reproduce the issue.

Select settings
Select model

Possible Solution

Context

Screenshots

ParisNeo commented 2 months ago

this is like the red ring of death of exllamav2. It hapens when sometghing went wrong wile installing it. I never actually managed to find why this hapens on some machines.

belaelekelemer commented 2 months ago

Similar issue on windows (hardware rtx 3060, p106-100) error message: DLL load failed while importing exllamav2_ext: The specified procedure could not be found. Its asked multiple times about pytorch force reinstall.

ParisNeo commented 1 month ago

You can try reinstalling exllamav2 manually by opening the lollms conda environment and doing a pip install