carlrobertoh / llm-client

User-friendly Java HTTP client that provides access to large language model APIs and services
https://central.sonatype.com/artifact/ee.carlrobert/llm-client
MIT License
17 stars 27 forks source link

Claude vision support #17

Closed moritzfl closed 5 months ago

moritzfl commented 5 months ago

This adds vision support for Claude.

Beware that this does break API compatibility due to polymorphism in message contents (which can now be either image or text for the current Claude API with Claude 3 models).

Usage is pretty straight forward and similar to the API description (https://docs.anthropic.com/claude/docs/vision):

ClaudeCompletionRequest request = new ClaudeCompletionRequest();
List<ClaudeMessageContent> messageContent = new ArrayList<>();
messageContent.add(new ClaudeMessageImageContent(new ClaudeBase64Source("image/jpeg", jpegBytes)));
messageContent.add(new ClaudeMessageTextContent("What is in the image?"));

ClaudeCompletionRequestMessage message = new ClaudeCompletionRequestMessage("user", messageContent);
request.setMessages(Collections.singletonList(message));
carlrobertoh commented 5 months ago

Nice! Thank you!