search

carlrobertoh / llm-client

User-friendly Java HTTP client that provides access to large language model APIs and services
https://central.sonatype.com/artifact/ee.carlrobert/llm-client
MIT License
21 stars 29 forks source link

OpenAI and Azure vision support #18

Closed moritzfl closed 7 months ago

moritzfl commented 7 months ago

This is pretty similar to "Claude vision support" #17

Similar to the Claude PR, this does break API compatibility due to polymorphism in message contents (which can now be either image or text).

API reference: https://platform.openai.com/docs/guides/vision

// Vision will only work with OpenAIChatCompletionModel.GPT_4_VISION_PREVIEW currently

List<OpenAIMessageContent> content = new ArrayList<>();

content.add(new OpenAIMessageImageURLContent(new OpenAIImageUrl("https://picsum.photos/200")));
// Or use a local image: new OpenAIImageUrl("image/jpeg", Files.readAllBytes(Paths.get("/path/to/local/image.jpg")))

content.add(new OpenAIMessageTextContent("What can you see in this picture?"));

OpenAIChatCompletionMessage message = new OpenAIChatCompletionMessage("user", content);
carlrobertoh commented 7 months ago

Awesome! Have you already started with the UI? If not, I could take a look sometime next week, if not sooner.

moritzfl commented 7 months ago

No - I have not done any work on the UI yet. I could look into it but I am busy during the easter holiday so it might take bit longer. Feel free to work on it.

Btw. after a meeting with the managing director we just informed all developers and technical writers at the company that I work for that they can from now on use CodeGPT with a dedicated API-Key provided through the company.

carlrobertoh commented 7 months ago

No - I have not done any work on the UI yet. I could look into it but I am busy during the easter holiday so it might take bit longer. Feel free to work on it.

Sure, I'll take a look.

Btw. after a meeting with the managing director we just informed all developers and technical writers at the company that I work for that they can from now on use CodeGPT with a dedicated API-Key provided through the company.

That's nice to hear! ❤️