spring-projects / spring-ai

An Application Framework for AI Engineering
https://docs.spring.io/spring-ai/reference/1.0-SNAPSHOT/index.html
Apache License 2.0
2.51k stars 605 forks source link

Add documentation how to use the OpenAI model client to access Groq inference engine. #996

Closed tzolov closed 2 days ago

tzolov commented 5 days ago

The Groq API provides high level of compatibility with the OpenAI API. The existing Spring AI OpenAI model client can be used to access the Groq inference API for most chat completion tasks. Provide documentation and instructions how to re-use the OpenAI Model Client for accessing the Groq chat completion endpoint.

KotlinFactory commented 4 days ago

Also how to access multiple inference engines at once

tzolov commented 3 days ago

@KotlinFactory can you please elaborate, what you mean by "multiple inference engines"?

Do you mean multiple Groq models or perhaps multiple Models across different providers?

Also can you provide some use cases to support such setup?