Function arguments parsing broken when using streaming

[widoriezebos]

When using functions combined with streaming completions, the ChatFunctionCallArgumentsSerializerAndDeserializer is used to deserialize arguments. This is broken because quotes get stripped from the chunks that get returned. This leads to a JSON deserialization exception of the combined chunks (when the arguments JSON gets parsed).

The token \":{\" is decoded to :{ and then decoded again, so losing all quotes in the process. This leads to a Jackson parse exception later when the aggregate of the chunks is (attempted to be) deserialised into the arguments.

I tried to use my own ChatFunctionCallMixIn but failed to plug that in. Also: there seems to be no way to override the mapper in OpenAiService which is kind of bummer because then I could have just plugged in a fix (in stead of patching the source below)

Would be great if this (or a better fix) is present in the next release. Thanks a bunch; really appreciate your work!

com/theokanning/openai/service/ChatFunctionCallArgumentsSerializerAndDeserializer.java

    @Override
    public JsonNode deserialize(JsonParser p, DeserializationContext ctxt) throws IOException {
      String json = p.getValueAsString();
      if (json == null || p.currentToken() == JsonToken.VALUE_NULL) {
        return null;
      }

      // ADDED
      // encode to valid JSON escape otherwise we will lose quotes
      json = MAPPER.writeValueAsString(json);
      // END ADDED

      try {
        JsonNode node = null;

        try {
          node = MAPPER.readTree(json);
        } catch (JsonParseException ignored) {
        }
        if (node == null || node.getNodeType() == JsonNodeType.MISSING) {
          node = MAPPER.readTree(p);
        }
        return node;
      } catch (Exception ex) {
                ex.printStackTrace();
        return null;
      }
    }
  }

[torbjorn-humane]

Just hit the same issue. Oh and I also second @widoriezebos comment that this is stellar work!

[balasaalin]

Hello! I also ran into the same issue.

[torbjorn-humane]

I ended up migrating to the azure open ai client library.