intelligentnode / IntelliNode

Access the latest AI models like ChatGPT, LLaMA, Diffusion, Gemini Hugging face, and beyond through a unified prompt layer and performance evaluation
https://show.intellinode.ai
Apache License 2.0
216 stars 14 forks source link

add hugging face inference support #5

Closed Barqawiz closed 1 year ago

Barqawiz commented 1 year ago

Create HuggingWrapper for the inference endpoint:

ENDPOINT = https://api-inference.huggingface.co/models/

curl example:

curl https://api-inference.huggingface.co/models/gpt2 \
        -X POST \
        -d '"Can you please let us know more details about your "' \
        -H "Authorization: Bearer ${API_TOKEN}"
# [{"generated_text":"Can you please let us know more details about your ids as a subscriber or other related project? Be sure to update your username and password or it will be stolen via email. Our information is only accessible through our website, and the payment support services"}]

Javascript example:

import fetch from "node-fetch";
async function query(data) {
    const response = await fetch(
        "https://api-inference.huggingface.co/models/gpt2",
        {
            headers: { Authorization: `Bearer ${API_TOKEN}` },
            method: "POST",
            body: JSON.stringify(data),
        }
    );
    const result = await response.json();
    return result;
}
query("Can you please let us know more details about your ").then((response) => {
    console.log(JSON.stringify(response));
});
// [{"generated_text":"Can you please let us know more details about your ids as a subscriber or other related project? Be sure to update your username and password or it will be stolen via email. Our information is only accessible through our website, and the payment support services"}]

detailed parameters: https://huggingface.co/docs/api-inference/detailed_parameters

Barqawiz commented 1 year ago

hugging face inference added to version 0.0.13