search

EvanZhouDev / gemini-ai

The simpler JavaScript Gemini SDK
https://www.npmjs.com/package/gemini-ai
GNU General Public License v3.0
95 stars 16 forks source link

[Feature request] Timeout option #15

Closed XInTheDark closed 4 weeks ago

XInTheDark commented 4 weeks ago

Right now with the latest version of the API, Gemini sometimes doesn't give a response at all, even after a few seconds of loading. However, If I retry the completion it immediately works again. This behaviour happens occasionally but it's pretty hard to reproduce, and hence a timeout parameter passed to ask() would be a really helpful fix.

I think here's a basic description:

  1. If the set timeout duration is up, then the request to the Gemini API is immediately stopped.
  2. If streaming is enabled, then the stream will instantly stop upon the timeout.
  3. If streaming is disabled, then nothing is returned.

Alternatively, an error could be thrown - this seems less preferable though, although it could give more information to the user so that they can e.g. retry the request.

This idea probably needs a bit more consideration regarding the details. Any feedback about this?

EvanZhouDev commented 4 weeks ago

This should be pretty easy to implement, though it can also be done on the user side. I'll consider it though.

EvanZhouDev commented 4 weeks ago

Upon further consideration, a time limit for the standard non-streaming ask() function may perhaps be a good idea. However, when it comes to streaming, a set time limit as you have described for the entire request is not a particularly efficient way to check whether or not a request is still working (it's better to check interval between each piece of data). In other words, I do not believe that this feature, at least the specifics that you have described, is preferable for every user, and as such shouldn't be necessarily included. I will be closing this for now, but if you have another idea, I would be glad to hear it.

XInTheDark commented 4 weeks ago

Hi, I can understand the complexity of doing this for streaming chats. Would it then be better to at least add a feature that tells the API to stop streaming and also stop the request? Currently I don't think there's any option to actually stop the stream, so it will keep running in the background even if the user disregards the results produced.

XInTheDark commented 4 weeks ago

One way might be to pass an AbortController (this is what node-fetch supports).

Or even better (IMO), there could just be a Gemini.stop() function that

This way, the user can implement time-outs in a flexible way while not complicating the API code.