radames
commented
10 months ago hi @Luke2642 , thanks for the amazing feedback!
I need to run mores tests on windows, are you using WSL?
Interesting is requirements.txt has --extra-index-url https://download.pytorch.org/whl/cu121 and pip install should pick that up.
I don't think triton works on windows, and would be the basis for torch.compile. You don't need it indeed.
Thanks for the the tip, yes the await asyncio.sleep() is a bit arbitrary and might be a bad idea if you have a good GPU to run at higher FPS. The same for the queue, there is an async issue there, where the inference time is slower than the queue process, I choose the cleanup the queue each time, but not sure what's the best solution to match inference with queue.
Btw, I'm working to combine all the scripts and the UI on a single, app, it would be much easier to maintain and add other features.
Luke2642
Fantastic work, really works well! Couple of issues and workarounds:
On windows I used instead of source venv/bin/activate, this:
venv\scripts\activate
Then I used:
pip install torch==2.1.0+cu121 -f https://download.pytorch.org/whl/torch_stable.html
to get xformers working properly. Still get a no triton error, which I ignored.
Then I was noticing huge webcam lag, so I modifed each of the files, changing the thread sleep time from 1/120 so 8ms to 333ms. I was only getting 3fps or so anyway:
await asyncio.sleep(40.0 / 120.0)
in app-controlnetlora.py or app-img2img.py... so change whichever you are using. It seemed to be queuing up many many frames without this, creating huge lag.
Couldn't get 0.0.0.0 to work, so launch I using this, perhaps should be the default, so the browser works on https://127.0.01:7860
uvicorn "app-controlnetlora:app" --host 127.0.0.1 --port 7860 --reload