search

Azure / azure-sdk-for-java

This repository is for active development of the Azure SDK for Java. For consumers of the SDK we recommend visiting our public developer docs at https://docs.microsoft.com/java/azure/ or our versioned developer docs at https://azure.github.io/azure-sdk-for-java.
MIT License
2.25k stars 1.93k forks source link

[OPENAI] Json parse exception while request OpenAI API in java #35734

Closed cuongnguyengit closed 12 months ago

cuongnguyengit commented 1 year ago

Hi Microsoft Dev Team, I had an exception when i requested to your openai azure api. I used the sample code final List messages = new ArrayList<>(); messages.add(new.ChatMessage(ChatRole.USER).setContent("tôi cần gì đâu"));

com.azure.ai.openai.models.ChatCompletionsOptions options = new com.azure.ai.openai.models.ChatCompletionsOptions(messages);
options.setModel("gpt-35-turbo");
options.setStream(false);
options.setN(1);
options.setMaxTokens(256);
options.setLogitBias(new HashMap<>());

ChatCompletions chatCompletions = client.getChatCompletions("***", options);

Exception or Stack Trace 09:14:44.921 [main] ERROR com.azure.core.implementation.serializer.DefaultJsonSerializer - com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'data': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') at [Source: (byte[])"data: {"id":"chatcmpl-7YPqqfztcROs341l4dOLDvFXvFOlv","object":"chat.completion.chunk","created":1688436880,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"role":"assistant"}}],"usage":null}data: {"id":"chatcmpl-7YPqqfztcROs341l4dOLDvFXvFOlv","object":"chat.completion.chunk","created":1688436880,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"content":"L"}}],"usage":null}data: {"id":"chatcmpl-7YPqqfztcROs341l4dOLDvFXvFOlv","object":"chat"[truncated 26688 bytes]; line: 1, column: 6] java.io.UncheckedIOException: com.fasterxml.jackson.core.JsonParseException: Unrecognized token 'data': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') at [Source: (byte[])"data: {"id":"chatcmpl-7YPqqfztcROs341l4dOLDvFXvFOlv","object":"chat.completion.chunk","created":1688436880,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"role":"assistant"}}],"usage":null}

data: {"id":"chatcmpl-7YPqqfztcROs341l4dOLDvFXvFOlv","object":"chat.completion.chunk","created":1688436880,"model":"gpt-35-turbo","choices":[{"index":0,"finish_reason":null,"delta":{"content":"L"}}],"usage":null}

data: {"id":"chatcmpl-7YPqqfztcROs341l4dOLDvFXvFOlv","object":"chat"[truncated 26688 bytes]; line: 1, column: 6] at com.azure.core.implementation.serializer.DefaultJsonSerializer.deserializeFromBytes(DefaultJsonSerializer.java:34) at com.azure.core.implementation.util.FluxByteBufferContent.toObject(FluxByteBufferContent.java:94) at com.azure.core.util.BinaryData.toObject(BinaryData.java:1241) at com.azure.core.util.BinaryData.toObject(BinaryData.java:1003) at com.azure.ai.openai.OpenAIClient.getChatCompletions(OpenAIClient.java:384) @

alzimmermsft commented 12 months ago

Thanks for filing this @cuongnguyengit.

@mssfang could you take a look at this.

mssfang commented 12 months ago

Thanks @alzimmermsft

@cuongnguyengit Thank you for reaching out to us. I won't be able to reproduce the error locally. Can you give more information, such as what version of SDK you are running and what is the complete sample you are using?

Given my local sample below and output for reference.

     OpenAIClient client = new OpenAIClientBuilder()
                .endpoint(endpoint)
                .credential(new AzureKeyCredential(azureOpenaiKey))
                .buildClient();

        final List messages = new ArrayList<>();
        messages.add(new ChatMessage(ChatRole.USER).setContent("tôi cần gì đâu"));
        ChatCompletionsOptions options = new ChatCompletionsOptions(messages)
                .setModel("gpt-35-turbo") // which can be removed.
                .setStream(false)
                .setN(1)
                .setMaxTokens(256)
                .setLogitBias(new HashMap<>());

        ChatCompletions chatCompletions = client.getChatCompletions("gpt-35-turbo", options);

        System.out.printf("Model ID=%s is created at %d.%n", chatCompletions.getId(), chatCompletions.getCreated());
        for (ChatChoice choice : chatCompletions.getChoices()) {
            ChatMessage message = choice.getMessage();
            System.out.printf("Index: %d, Chat Role: %s.%n", choice.getIndex(), message.getRole());
            System.out.println("Message:");
            System.out.println(message.getContent());
        }

        System.out.println();
        CompletionsUsage usage = chatCompletions.getUsage();
        System.out.printf("Usage: number of prompt token is %d, "
                        + "number of completion token is %d, and number of total tokens in request and response is %d.%n",
                usage.getPromptTokens(), usage.getCompletionTokens(), usage.getTotalTokens());

Output:

Model ID=chatcmpl-7Z3fwFiC3WNw0GE5QPi0WVQgsZiBI is created at 1688589964.
Index: 0, Chat Role: assistant.
Message:
Là trợ lý ảo, tôi rất mong muốn được hỗ trợ bạn. Vui lòng cho tôi biết cần hỗ trợ gì, tôi sẽ cố gắng hết sức để giúp đỡ bạn.

Usage: number of prompt token is 18, number of completion token is 68, and number of total tokens in request and response is 86.
github-actions[bot] commented 12 months ago

Hi @cuongnguyengit. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

cuongnguyengit commented 12 months ago

Vào Th 5, 6 thg 7, 2023 lúc 03:46 Shawn Fang @.***> đã viết:

Thanks @alzimmermsft https://github.com/alzimmermsft

@cuongnguyengit https://github.com/cuongnguyengit Thank you for reaching out to us. I won't be able to reproduce the error locally. Can you give more information, such as what version of SDK you are running and what is the complete sample you are using?

Given my local sample below and output for reference.

 OpenAIClient client = new OpenAIClientBuilder()
            .endpoint(endpoint)
            .credential(new AzureKeyCredential(azureOpenaiKey))
            .buildClient();

    final List messages = new ArrayList<>();
    messages.add(new ChatMessage(ChatRole.USER).setContent("tôi cần gì đâu"));
    ChatCompletionsOptions options = new ChatCompletionsOptions(messages)
            .setModel("gpt-35-turbo") // which can be removed.
            .setStream(false)
            .setN(1)
            .setMaxTokens(256)
            .setLogitBias(new HashMap<>());

    ChatCompletions chatCompletions = client.getChatCompletions("gpt-35-turbo", options);

    System.out.printf("Model ID=%s is created at %d.%n", chatCompletions.getId(), chatCompletions.getCreated());
    for (ChatChoice choice : chatCompletions.getChoices()) {
        ChatMessage message = choice.getMessage();
        System.out.printf("Index: %d, Chat Role: %s.%n", choice.getIndex(), message.getRole());
        System.out.println("Message:");
        System.out.println(message.getContent());
    }

    System.out.println();
    CompletionsUsage usage = chatCompletions.getUsage();
    System.out.printf("Usage: number of prompt token is %d, "
                    + "number of completion token is %d, and number of total tokens in request and response is %d.%n",
            usage.getPromptTokens(), usage.getCompletionTokens(), usage.getTotalTokens());

Output:

Model ID=chatcmpl-7Z3fwFiC3WNw0GE5QPi0WVQgsZiBI is created at 1688589964. Index: 0, Chat Role: assistant. Message: Là trợ lý ảo, tôi rất mong muốn được hỗ trợ bạn. Vui lòng cho tôi biết cần hỗ trợ gì, tôi sẽ cố gắng hết sức để giúp đỡ bạn.

Usage: number of prompt token is 18, number of completion token is 68, and number of total tokens in request and response is 86.

— Reply to this email directly, view it on GitHub https://github.com/Azure/azure-sdk-for-java/issues/35734#issuecomment-1622478324, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKZGI7OON63B7GTRCMYUR53XOXHDVANCNFSM6AAAAAAZ5PCQXQ . You are receiving this because you were mentioned.Message ID: @.***>

Sorry, I found a bug in my code and had no choice send it to you. I set True for stream.

Thanks for your support.

mssfang commented 12 months ago

@cuongnguyengit Please don't use setStream() method, which will be deleted in the future. Please use the streaming API instead.

See sample: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetChatCompletionsStreamSample.java#L50

In beta.2 version, we have a garbled character bug that could be fixed in https://github.com/Azure/azure-sdk-for-java/pull/35641

github-actions[bot] commented 12 months ago

Hi @cuongnguyengit. Thank you for opening this issue and giving us the opportunity to assist. To help our team better understand your issue and the details of your scenario please provide a response to the question asked above or the information requested above. This will help us more accurately address your issue.

cuongnguyengit commented 12 months ago

Vào Th 5, 6 thg 7, 2023 lúc 06:25 Shawn Fang @.***> đã viết:

@cuongnguyengit https://github.com/cuongnguyengit Please don't use setStream() method, which will be deleted in the future. Please use the streaming API instead.

See sample: https://github.com/Azure/azure-sdk-for-java/blob/main/sdk/openai/azure-ai-openai/src/samples/java/com/azure/ai/openai/usage/GetChatCompletionsStreamSample.java#L50

— Reply to this email directly, view it on GitHub https://github.com/Azure/azure-sdk-for-java/issues/35734#issuecomment-1622670417, or unsubscribe https://github.com/notifications/unsubscribe-auth/AKZGI7LHSHYHBD6JDIK4OVDXOXZU3ANCNFSM6AAAAAAZ5PCQXQ . You are receiving this because you were mentioned.Message ID: @.***>

Maybe I used old docs. Thanks.