Longer timeout limit needed for LeMUR (java.net.SocketTimeoutException: timeout)

[smithakolan]

Transcripts with longer token size time out with LeMUR functions.

Requests with longer files result in this error: Caused by: java.net.SocketTimeoutException: timeout

For context, here is the code used:

import com.assemblyai.api.AssemblyAI;
import com.assemblyai.api.resources.lemur.requests.*;
import com.assemblyai.api.resources.transcripts.types.*;
import java.util.List;

public final class App {
    public static void main(String[] args) {

        AssemblyAI client = AssemblyAI.builder()
                .apiKey("api-key")
                .build();

        String url = "https://storage.googleapis.com/aai-web-samples/meeting.mp3";

        Transcript transcript = client.transcripts().transcribe(url);

        String prompt = "Provide a brief summary of the transcript.";

        var params = LemurTaskParams.builder()
                .prompt(prompt)
                .transcriptIds(List.of(transcript.getId()))
                .build();

        var result = client.lemur().task(params);

        System.out.println(result.getResponse());

[Swimburger]

I ran into the same issue and increased the timeout to 30s. Should we change it to 60s, maybe? Is that a sane default for the model we support?

In the future, we should use this larger timeout for only LeMUR, as the other API endpoints shouldn't ever take longer than a second.

[dsinghvi]

@Swimburger is planning to check what you do for the other SDKs.

On the Fern side, we can do 2 things (unclear how helpful):

• Set timeout to infinity
• Generate a RequestOptions and allow the user to override
• OpenAPI extension to set timeout per endpoint

[Swimburger]

"In the Python SDK we set it to unlimited and by giving the user an option to specify a custom timeout. That's the only way to make it work and configurable at the same time."

Node SDK doesn't have a timeout configured but uses fetch, and each runtime has a different default timeout or no timeout for fetch.

[fnikitin]

Hi, I am currently testing your java client and I get exactly this exception for LeMur and also when uploading my audio to your server. Can you explain me why you set this kind of limit at the level of the java client knowing that the python client (that I have also tested and work as expected) has no time limitation ? I don't understand the reasoning here - if you want to have constraint somewhere it should be in your backend not in the clients. Imho @dsinghvi comments give some nice solutions that would be really appreciated. In the meantime I cannot upload anything ... I could create my own client to upload my files on your server but it would be a waste of time. Cheers.

[Swimburger]

@fnikitin Sorry for the inconvenience. We're generating our SDKs using Fern. Fern hasn't ran into this specialized use case, so they need to do some work to enable us to control timeouts on a per endpoint basis. For now, we'll configure the timeouts manually to unlock this while Fern works on a solution in their generator.