turboderp / exui

Web UI for ExLlamaV2
MIT License
449 stars 43 forks source link

server.py won't open #53

Open ReMeDy-TV opened 7 months ago

ReMeDy-TV commented 7 months ago

This is in Windows 10 with Python 3.10 on CUDA 12.1. It seems to complain about torch, but I installed the appropriate pytorch here. This is my first time attempting to run EXUI. I believe I followed all the install instructions. Should I upgrade my Python, although 3.10 should work. I don't have any issues running Ooba by comparison.

E:\exui>python server.py
Traceback (most recent call last):
  File "E:\exui\server.py", line 9, in <module>
    import torch
  File "C:\Users\David\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torch\__init__.py", line 141, in <module>
    raise err
OSError: [WinError 126] The specified module could not be found. Error loading "C:\Users\David\AppData\Local\Packages\PythonSoftwareFoundation.Python.3.10_qbz5n2kfra8p0\LocalCache\local-packages\Python310\site-packages\torch\lib\shm.dll" or one of its dependencies.
Polkadoty commented 6 months ago

I have this same issue but with Windows 11 and Python 3.11. I can't figure out where this module error is from, because I have CUDA, pytorch, and the wheel corresponding to that cuda and pytorch both installed correctly.

ramzeez88 commented 2 weeks ago

Guys ,first go and install torch latest stable version from their website.( around 2.5GB) After you have it installed , go to here and find the version that suits your python version and cuda. When you do copy the link and in your terminal type pip install and your copied link. Then you need to git clone the exui repo git clone https://github.com/turboderp/exui
After the exui folder is created you need to change your terminal directory to it : cd exui. Then pip nstall pynvml Flask waitress tokenizers Next type in terminal python server.py Also pay attention to the terminal for errors, in case there are more requirements needed to pip install. This is how I did it.