vacuityv / ai-java-sdk

java sdk for ai apis. Support claude ai and google gemini and part of openai (chatgpt). 支持chatgpt/gemini/claude的java sdk。
29 stars 9 forks source link

接口添加图片解析功能 #3

Closed wkywj1 closed 7 months ago

wkywj1 commented 7 months ago

大佬好,claude3模型支持图片识别,代码里能不能加上这个功能,谢谢!

vacuityv commented 7 months ago

理论上现在就是支持的,你可以用以下代码测试一下:

@Test
public void vision() throws IOException {
        String imagePath = "/Users/vacuity/Downloads/sonatype.jpg";
        // 读取图片文件
        byte[] imageBytes = Files.readAllBytes(Paths.get(imagePath));

        // 将图片文件转换为Base64编码
        String base64Image = Base64.getEncoder().encodeToString(imageBytes);

        ClaudeClient client = new ClaudeClient(API_KEY);
        List<ChatMessage> messages = new ArrayList<>();

        ChatMessageContent content = new ChatMessageContent();
        ChatMessageContent.ContentSource source = new ChatMessageContent.ContentSource();
        source.setType("base64");
        source.setMediaType("image/jpeg");
        source.setData(base64Image);

        content.setType("image");
        content.setSource(source);
        ChatMessageContent content2 = new ChatMessageContent();
        content2.setType("text");
        content2.setText("what is this?");

        ChatMessage chatMessage = new ChatMessage("user", Arrays.asList(content, content2));
        messages.add(chatMessage);

        ChatRequest request = ChatRequest.builder()
                .model("claude-3-opus-20240229")
                .messages(messages)
                .maxTokens(1024)
                .build();
        try {
            ChatResponse response = client.chat(request);
            System.out.println(response);
        } catch (VacSdkException e) {
            if (e.getDetail() != null) {
                System.out.println(e.getDetail().getError().getMessage());
            }
        }
    }
vacuityv commented 7 months ago

@wkywj1 我本地测试是没问题的,你可以试下,如果网络不太好的话可以在创建client的时候把超时时间设长一点