search

huggingface / huggingface.js

Utilities to use the Hugging Face Hub API
https://hf.co/docs/huggingface.js
MIT License
1.37k stars 207 forks source link

Incompatibility between OpenAI and HF's Chat Completion `response_format` #932

Open sadra-barikbin opened 1 day ago

sadra-barikbin commented 1 day ago

Hi there!

OpenAI and HF api seem to have diverged on chat_completion response_format. That of the OpenAI has a type attribute which takes one of three values text, json_object (free-form JSON generation which isn't in TGI yet) and json_schema, and for the last one it has another json_schema attribute which itself contains name, schema, description and strict attributes.

On the other hand, HF's response_format: ChatCompletionInputGrammarType contains type and value attributes.

This might break the OpenAI api usability on HF models. Could we have compatibility between these two on this matter? Or does it need probably breaking changes in TGI and huggingface_hub?

@Wauplin

Wauplin commented 1 day ago

Cc @drbh @Narsil @OlivierDehaene on the TGI side