Open Sam7 opened 1 year ago
I am also having this error sometimes
could someone change it? if yes, please write in telegram, help @volume_k
+1
We need to accept CancellationToken on async functions and pass it to the HTTP requests.
a vote for making this user settable. I'm working with some big contexts in GPT-4 and they can take awhile. I have verified in OpenAI's Playground that in GPT-4 chat mode my prompts take well over 100 seconds to successfully complete.
I download the repository and made an ad-hoc fix, increasing the timeout to 1000 seconds. I rebuilt the nuget file and installed it locally. I tested it on my application and it works. At temporary fix to the problem. Much better would be a user parameter when creating a chat or other session.
in EndpointBase.cs:
protected HttpClient GetClient() { if (_Api.Auth?.ApiKey is null) { throw new AuthenticationException("You must provide API authentication. Please refer to https://github.com/OkGoDoIt/OpenAI-API-dotnet#authentication for details."); }
HttpClient client;
var clientFactory = _Api.HttpClientFactory;
if (clientFactory != null)
{
client = clientFactory.CreateClient();
}
else
{
client = new HttpClient();
}
client.DefaultRequestHeaders.Authorization = new System.Net.Http.Headers.AuthenticationHeaderValue("Bearer", _Api.Auth.ApiKey);
// Further authentication-header used for Azure openAI service
client.DefaultRequestHeaders.Add("api-key", _Api.Auth.ApiKey);
client.DefaultRequestHeaders.Add("User-Agent", UserAgent);
// Jim Rutt increased timeout to 1000
client.Timeout = TimeSpan.FromSeconds(1000);
if (!string.IsNullOrEmpty(_Api.Auth.OpenAIOrganization)) client.DefaultRequestHeaders.Add("OpenAI-Organization", _Api.Auth.OpenAIOrganization);
return client;
}
Why not instantiate your own HttpClient with whatever timeout settings you want, and pass that into the existing service constructors?
Why not instantiate your own HttpClient with whatever timeout settings you want, and pass that into the existing service constructors?
Do you have an example of this?
Of what? chatGPT can show you how to instantiate an HttpClient and set its timeout settings. Then you pass it into the constructor that accepts an HttpClient reference
For people that are new to .NET like me, this is how I did it in C# (Warning, I have no idea what I'm doing and chatgpt did most of this work )
Add a reference to System.Net.Http to your project
Add the following lines in the right places in your project:
using System.Net.Http;
public class CustomHttpClientFactory : IHttpClientFactory
{
public HttpClient CreateClient(string name)
{
return new HttpClient
{
Timeout = TimeSpan.FromSeconds(300)
};
}
}
OpenAIAPI api = new OpenAIAPI(myApiKey)
{
HttpClientFactory = new CustomHttpClientFactory()
}
I have sort of the opposite issue. Some times the call hangs for 100 seconds and that's a long time if the answer is not coming. So I would need a timeout of maybe 10-15 seconds but only if it hasn't started answering yet. As soon as it is answering, then there should not be any timeout in my opinion.
I have sort of the opposite issue. Some times the call hangs for 100 seconds and that's a long time if the answer is not coming. So I would need a timeout of maybe 10-15 seconds but only if it hasn't started answering yet. As soon as it is answering, then there should not be any timeout in my opinion.
Is there a way to understand that CreateChatCompletionAsync
writing an answer or just stuck? If so maybe we can keep the time with Stopwatch
and cancel the request manually.
Perhaps use StreamCompletionEnumerableAsync
instead, so that you'd know if it was in the process of responding, versus just waiting for anything at all to come back?
Do we have any fix for this?
I'm not sure this is a bug to be fixed. The SDK allows you to pass in your own HttpClient with your own preferences for timeouts, etc: https://github.com/OkGoDoIt/OpenAI-API-dotnet/issues/102#issuecomment-1563708070
The custom HTTP client is the "right" way to do this, but I should probably add a helper to make it easy to set a custom timeout and take care of this behind the scenes. The custom HTTP client is a bit clunky.
I often get this error. But other times it works without issues. Any ideas on how to change the timeout? Has anyone else had this issue?