search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
40.57k stars 5.31k forks source link

responses #5355

Closed Alastor1933 closed 7 months ago

Alastor1933 commented 9 months ago

Describe the bug

idk its bug or im dumb or else if im trying 7B model AWQ on Nvidia Quadro M4000 and i5-13600K it starts im loading model everything works but responses from chat is blank it says chat typing but there is no response for questions or else

Is there an existing issue for this?

Reproduction

  1. git clone oobabooga from github to folder
  2. turn off antivirus
  3. open start_windows.bat
  4. enter oobabooga gui in browser
  5. install model from TheBloke
  6. load model

Screenshot

obraz_2024-01-23_105058478 obraz_2024-01-23_105146574

Logs

don't get any logs just blank response

System Info

Nvidia Quadro M4000 and i5-13600K
bartowski1182 commented 9 months ago

I don't think the M4000 works with AWQ

according to: https://github.com/casper-hansen/AutoAWQ

you need Turing or newer

github-actions[bot] commented 7 months ago

This issue has been closed due to inactivity for 2 months. If you believe it is still relevant, please leave a comment below. You can tag a developer in your comment.