search

oobabooga / text-generation-webui

A Gradio web UI for Large Language Models.
GNU Affero General Public License v3.0
39.42k stars 5.18k forks source link

Lag/delay when clicking buttons in the gradio UI #4138

Open LoganDark opened 11 months ago

LoganDark commented 11 months ago

Describe the bug

When clicking buttons in the gradio web interface, there's a noticeable delay before the button press is actually received by the python server. I am not sure whether this delay is inherent to gradio or solvable in any way, but it severely hampers one use-case: repeatedly generating with max tokens = 1 in order to closely supervise / collaborate with the model. So if there is any way to reduce or eliminate this delay that would be really nice

Is there an existing issue for this?

Reproduction

Have your terminal on-screen, and click a button in the interface that usually results in console output (i.e. generate button, saving preset, etc). Not only will the web interface take a second to show the orange outline, but it will also take a second for anything to show up in console.

Devtools shows that the server takes around 400ms to respond to even a simple "stop" request when it isn't even generating anything. Maybe a python profiler should be taken to this.

Screenshot

No response

Logs

N/A

System Info

Operating System: Windows 11 Enterprise 64-bit (10.0, Build 22621)
Language: English (Regional Setting: English)
System Manufacturer: Micro-Star International Co., Ltd.
System Model: MS-7D97
BIOS: 1.20
Processor: 12th Gen Intel(R) Core(TM) i5-12400F (12 CPUs), ~5.2GHz
Memory: 16384MB RAM
Page file: 45178MB used, 3833MB available

Name: NVIDIA GeForce RTX 3060
Manufacturer: NVIDIA
Chip Type: NVIDIA GeForce RTX 3060
DAC Type: Integrated RAMDAC
Device Type: Full Display Device
Approx. Total Memory: 20250 MB
Display Memory (VRAM): 12129 MB
Shared Memory: 8121 MB
Current Display Mode: 3840 x 2160 (32 bit) (60Hz)
Monitor: Generic PnP Monitor
HDR: Supported
LoganDark commented 3 months ago

Not stale.

LoganDark commented 3 months ago

Not stale.

LoganDark commented 2 months ago

Not stale.

LoganDark commented 2 months ago

Not stale.

LoganDark commented 2 months ago

Not stale.

LoganDark commented 2 months ago

Not stale.

LoganDark commented 1 month ago

Not stale.

LoganDark commented 1 month ago

Not stale.

LoganDark commented 1 month ago

Not stale.

LoganDark commented 1 month ago

Not stale.

LoganDark commented 1 month ago

Not stale.

LoganDark commented 3 weeks ago

Not stale.

LoganDark commented 2 weeks ago

Not stale.

LoganDark commented 1 week ago

Not stale.

LoganDark commented 4 days ago

Not stale.