When clicking buttons in the gradio web interface, there's a noticeable delay before the button press is actually received by the python server. I am not sure whether this delay is inherent to gradio or solvable in any way, but it severely hampers one use-case: repeatedly generating with max tokens = 1 in order to closely supervise / collaborate with the model. So if there is any way to reduce or eliminate this delay that would be really nice
Is there an existing issue for this?
[X] I have searched the existing issues
Reproduction
Have your terminal on-screen, and click a button in the interface that usually results in console output (i.e. generate button, saving preset, etc). Not only will the web interface take a second to show the orange outline, but it will also take a second for anything to show up in console.
Devtools shows that the server takes around 400ms to respond to even a simple "stop" request when it isn't even generating anything. Maybe a python profiler should be taken to this.
Screenshot
No response
Logs
N/A
System Info
Operating System: Windows 11 Enterprise 64-bit (10.0, Build 22621)
Language: English (Regional Setting: English)
System Manufacturer: Micro-Star International Co., Ltd.
System Model: MS-7D97
BIOS: 1.20
Processor: 12th Gen Intel(R) Core(TM) i5-12400F (12 CPUs), ~5.2GHz
Memory: 16384MB RAM
Page file: 45178MB used, 3833MB available
Name: NVIDIA GeForce RTX 3060
Manufacturer: NVIDIA
Chip Type: NVIDIA GeForce RTX 3060
DAC Type: Integrated RAMDAC
Device Type: Full Display Device
Approx. Total Memory: 20250 MB
Display Memory (VRAM): 12129 MB
Shared Memory: 8121 MB
Current Display Mode: 3840 x 2160 (32 bit) (60Hz)
Monitor: Generic PnP Monitor
HDR: Supported
Describe the bug
When clicking buttons in the gradio web interface, there's a noticeable delay before the button press is actually received by the python server. I am not sure whether this delay is inherent to gradio or solvable in any way, but it severely hampers one use-case: repeatedly generating with max tokens = 1 in order to closely supervise / collaborate with the model. So if there is any way to reduce or eliminate this delay that would be really nice
Is there an existing issue for this?
Reproduction
Have your terminal on-screen, and click a button in the interface that usually results in console output (i.e. generate button, saving preset, etc). Not only will the web interface take a second to show the orange outline, but it will also take a second for anything to show up in console.
Devtools shows that the server takes around 400ms to respond to even a simple "stop" request when it isn't even generating anything. Maybe a python profiler should be taken to this.
Screenshot
No response
Logs
N/A
System Info