search

matlab-deep-learning / llms-with-matlab

Connect MATLAB to LLM APIs, including OpenAI® Chat Completions, Azure® OpenAI Services, and Ollama™
Other
97 stars 21 forks source link

Support function calls in streaming #10

Closed toshiakit closed 6 months ago

toshiakit commented 6 months ago

Made changes to +llms/+internal/callOpenAIChatAPI.m

pat = '{"' + wildcardPattern + '":';
if contains(streamedText,pat)
    s = jsondecode(streamedText);
    if contains(s.function.arguments,pat)
        prompt = jsondecode(s.function.arguments);
        s.function.arguments = prompt;
    end
    message = struct("role", "assistant", ...
         "content",[], ...
         "tool_calls",jsondecode(streamedText));
else
    message = struct("role", "assistant", ...
        "content", streamedText);
end

+llms/+stream/responseStreamer.m

try
    json = jsondecode(str{i});
catch ME
    error("API returned il-formed json: " + str{i})
end