Closed synw closed 1 year ago
yeah, that's a great point. I just added the ability to configure this parameter (but not disable entirely) in 1.3.14:
if you think being able to disable it is a must-have, I can put that in as well. otherwise, hopefully this works well for you.
It works, thanks. We don't need a disable option, it's fine
Hi, thanks for this nice extension
The timeout is currently hard-coded to 1 minute. I would like to be able to make it longer. My use case: I am doing experiments using small local ggml models running via a Llama.cpp based Go server of my own (something like LocalAi). Because I have no gpu when the prompt is long it takes a lot of time to process it with only cpu, making the timeout to be reached but my local server has not finished to respond with an inference result
It would be great to have a timeout parameter in the settings to increase the timeout if necessary, or even to disable it (there is a cancel request button anyway)