search

nvms / wingman

Your pair programming wingman. Supports OpenAI, Anthropic, or any LLM on your local inference server.
https://marketplace.visualstudio.com/items?itemName=nvms.ai-wingman
ISC License
61 stars 10 forks source link

Configurable timeout #11

Closed synw closed 1 year ago

synw commented 1 year ago

Hi, thanks for this nice extension

The timeout is currently hard-coded to 1 minute. I would like to be able to make it longer. My use case: I am doing experiments using small local ggml models running via a Llama.cpp based Go server of my own (something like LocalAi). Because I have no gpu when the prompt is long it takes a lot of time to process it with only cpu, making the timeout to be reached but my local server has not finished to respond with an inference result

It would be great to have a timeout parameter in the settings to increase the timeout if necessary, or even to disable it (there is a cancel request button anyway)

nvms commented 1 year ago

yeah, that's a great point. I just added the ability to configure this parameter (but not disable entirely) in 1.3.14:

Screenshot 2023-08-15 at 1 24 19 PM

if you think being able to disable it is a must-have, I can put that in as well. otherwise, hopefully this works well for you.

synw commented 1 year ago

It works, thanks. We don't need a disable option, it's fine