search

spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/1.0-SNAPSHOT/index.html
Apache License 2.0
3.11k stars 776 forks source link

Common types for text-to-speech #1496

Open habuma opened 1 week ago

habuma commented 1 week ago

Similar to what I suggested in #1478, it would be great if text-to-speech had a set of common types. The SpeechModel interface, as well as SpeechPrompt, SpeechResponse, and StreamingSpeechModel feel like what I'd expect with such types, but they are currently delivered in the OpenAI module. Even though OpenAI is the only implementation, it feels like those types should be in core with the implementations and OpenAI-specific extensions in the OpenAI module.

Moreover, while SpeechPrompt feels like it should be in core, it carries OpenAiAudioSpeechOptions. Perhaps there should be a more generic SpeechOptions that is carried by SpeechPrompt, with OpenAiAudioSpeechOptions being an extension of SpeechOptions.

Altogether, this would not only make the types more consistent with how the types for chat and other models are structured, it also sets the stage for additional text-to-speech implementations should more APIs that offer that be added to Spring AI.

ThomasVitale commented 1 week ago

I like the suggestion and I'm available to work on this. I'll have a PR ready soon.

mudabirhussain commented 1 week ago

Similar to what I suggested in #1478, it would be great if text-to-speech had a set of common types. The SpeechModel interface, as well as SpeechPrompt, SpeechResponse, and StreamingSpeechModel feel like what I'd expect with such types, but they are currently delivered in the OpenAI module. Even though OpenAI is the only implementation, it feels like those types should be in core with the implementations and OpenAI-specific extensions in the OpenAI module.

Moreover, while SpeechPrompt feels like it should be in core, it carries OpenAiAudioSpeechOptions. Perhaps there should be a more generic SpeechOptions that is carried by SpeechPrompt, with OpenAiAudioSpeechOptions being an extension of SpeechOptions.

Altogether, this would not only make the types more consistent with how the types for chat and other models are structured, it also sets the stage for additional text-to-speech implementations should more APIs that offer that be added to Spring AI.

What is the reason for having SpeechResponse in the core, since it is already an implementation of ModelResponse, which belongs to the core family? Most of the classes or interfaces mentioned above are already implementations or extensions of existing core family classes or interfaces.