Huggingface Endpoint Integration Request.

[Niansuh]

Could you include both the Hugging Face endpoint and the Hugging Cloud dedicated endpoint, please?

[Calcium-Ion]

Could you please provide the documentation links for both the Hugging Face endpoint and the Hugging Face Cloud dedicated endpoint?

[Niansuh]

@Calcium-Ion

Serverless Inference API

https://huggingface.co/docs/api-inference/quicktour https://huggingface.co/docs/api-inference

Inference Endpoints (dedicated)

https://huggingface.co/docs/inference-endpoints/en/index https://ui.endpoints.huggingface.co

Serverless Inference API Example

-H "Authorization: Bearer hf_xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx" \
-H 'Content-Type: application/json' \
-d '{
    "model": "mistralai/Mistral-Nemo-Instruct-2407",
    "messages": [{"role": "user", "content": "What is the capital of France?"}],
    "max_tokens": 500,
    "stream": false
}'```

[Niansuh]

Inference Endpoints (dedicated) Example

# If necessary, install the openai Python library by running 
# pip install openai

from openai import OpenAI

client = OpenAI(
    base_url="https://hlb2vp60a9kpy2lu.us-east-1.aws.endpoints.huggingface.cloud/v1/", 
    api_key="hf_niansuhai"
)

chat_completion = client.chat.completions.create(
    model="tgi",
    messages=[
        {
            "role": "user",
            "content": "What is deep learning?"
        }
    ],
    stream=True,
    max_tokens=20
)

for message in chat_completion:
    print(message.choices[0].delta.content, end="")```

[Niansuh]

Also, include default system prompts in each channel.