RegisterAiService currently allows adding a chatLanguageModelSupplier. Unfortunately, I could not find a straightforward way of implementing a custom Supplier<StreamingChatLanguageModel>.
I'd suggest adding an additional property streamingChatLanguageModelSupplier to RegisterAiService, so we can use custom models.
I could not find a viable alternative for providing my own implementation, except for writing my own extension (like in quarkus-langchain4j-azure-openai), which seems to be overkill for that task.
I also think quarkus-langchain4j would profit from that feature, as it would allow to add more test cases with StreamingChatLanguageModels, when a custom model can be implemented like in:
public class StreamingTest {
static class MyModelSupplier implements Supplier<StreamingChatLanguageModel> {
@Override
public StreamingChatLanguageModel get() {
return (messages, handler) -> {
handler.onComplete(new Response<>(new AiMessage("42")));
};
}
}
@RegisterAiService(streamingChatLanguageModelSupplier = MyModelSupplier.class)
interface MyService {
@UserMessage("What is the Answer to the Ultimate Question of Life, the Universe, and Everything?")
Multi<String> chat();
}
// ...
}
RegisterAiService
currently allows adding achatLanguageModelSupplier
. Unfortunately, I could not find a straightforward way of implementing a customSupplier<StreamingChatLanguageModel>
.I'd suggest adding an additional property
streamingChatLanguageModelSupplier
toRegisterAiService
, so we can use custom models.I could not find a viable alternative for providing my own implementation, except for writing my own extension (like in
quarkus-langchain4j-azure-openai
), which seems to be overkill for that task.I also think
quarkus-langchain4j
would profit from that feature, as it would allow to add more test cases withStreamingChatLanguageModel
s, when a custom model can be implemented like in: