Truly `openai` drop-in replacement for chat completion

[Wauplin]

_Originally from @philschmid on slack (private):_

Being openai compatible for serving has become a standard so its awesome that TGI is. This also means what we can use openai sdk and it allows us to communicate that you can basically switch from closed source to open source with changing 1 variable (url). A lot of people are familiar with the openai already so the closer you are the easier it is for them to understand and switch. I get that when we have new features we cannot use openai but it might have been cool that we are in parity for everything existing, e.g.

from huggingface_hub import InferenceClient

client = InferenceClient(
  base_url='http://huggingface.co/api/integrations/dgx',
  api_key='hf_xx',

)
resp = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant." },
        {"role": "user", "content": "Count to 50"}
    ],
    stream=False,
    max_tokens=1024
)

print(resp.choices[0].message.content)

or

from huggingface_hub import InferenceClient

client = InferenceClient()

resp = client.chat.completions.create(
    model="meta-llama/Meta-Llama-3-8B-Instruct",
    messages=[
        {"role": "system", "content": "You are a helpful assistant." },
        {"role": "user", "content": "Count to 50"}
    ],
    stream=False,
    max_tokens=1024
)

print(resp.choices[0].message.content)

---

Note: to do that we'll need to:

• [ ] rename model to base_url
• [ ] rename model_id to model
• [ ] rename token to api_key
• [ ] add alias for client.chat.completions.create

[lappemic]

I would love to work on this @Wauplin.

To clarify:

• Would you rename the parameters of all the methods (also image_segmentation, image_to_image, etc.) or just the ones related to chat completions?
• It needs to be adjusted mostly in src/huggingface_hub/inference/_client.py. Is this correct?

[Wauplin]

Hi @lappemic, thanks for proposing yourself! For now I've created the issue but I'm not sure if we want and how we want to proceed. I've started to test things locally to see how the changes would looks like (API-wise but also internally). I think we will introduce changes only for openai compatibility but keep the current naming for most/all existing tasks without breaking changes. Changes will happen in both _client.py and the async client (automatically).

[lappemic]

Thanks for the feedback.

Ok, let me know if you decided on how to proceed and if you would like some help!

[Wauplin]

In the end I opened https://github.com/huggingface/huggingface_hub/pull/2384 with the corresponding changes. Still need to be discussed on new args