quarkiverse / quarkus-langchain4j

Quarkus Langchain4j extension
https://docs.quarkiverse.io/quarkus-langchain4j/dev/index.html
Apache License 2.0
116 stars 65 forks source link

vertexai.gemini does not respect timeout #638

Closed exaV closed 1 month ago

exaV commented 1 month ago

I get the following timeout exception when invoking gemini:

jakarta.ws.rs.ProcessingException: The timeout period of 10000ms has been exceeded while executing POST /v1/projects/myrealproject/locations/us-west1/publishers/google/models/gemini-1.5-flash:generateContent for server null

I tried the following configuration but it seems to have no effect:

quarkus.langchain4j.timeout=30s

I also tried adding a org.eclipse.microprofile.faulttolerance.Timeout annotation to my @RegisterAiService but that has no effect either. How do I configure the timeout for my an AiService backed by gemini? I could not find any properties in devui or in the documentation.

I found this bit of code in the implementation, which to me suggests that it is simply not configurable yet and does not inherit the default either, but I might be wrong.

geoand commented 1 month ago

That indeed looks like the case. Would you like to provide a fix for this?

Thanks