Kardbord / hfapigo

Unofficial (Golang) Go bindings for the Hugging Face Inference API
MIT License
61 stars 5 forks source link

Some models reject multiple inputs #35

Open Kardbord opened 7 months ago

Kardbord commented 7 months ago

Despite the docs indicating that multiple inputs are supported for several request types, some models only support a single input at a time. Support should be added to this library for making requests with either a list of inputs or a single input in order to support models that do not like lists.

Example:

# 1) Does not work
curl https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-8B-Instruct  \
    -X POST \
    -d '{"inputs": ["The quick brown fox", "jumps over the lazy dog"]}'  \
    -H "Authorization: Bearer ${HUGGING_FACE_TOKEN}" \
    -H 'Content-Type: application/json'

# 2) Works
curl https://api-inference.huggingface.co/models/meta-llama/Meta-Llama-3-8B-Instruct  \
    -X POST \
    -d '{"inputs": "The quick brown fox"}'  \
    -H "Authorization: Bearer ${HUGGING_FACE_TOKEN}" \
    -H 'Content-Type: application/json'

# 3) Works
curl https://api-inference.huggingface.co/models/gpt2 \
    -X POST  \
    -d '{"inputs": ["The quick brown fox", "jumps over the lazy dog"]}'  \
    -H "Authorization: Bearer ${HUGGING_FACE_TOKEN}" \
    -H 'Content-Type: application/json'

1) returns

Failed to deserialize the JSON body into the target type: inputs: invalid type: sequence, expected a string at line 1 column 11

2) returns

[
  {
    "generated_text": "The quick brown fox jumped over the lazy dog. The lazy dog chased after the quick brown fox. The quick brown fox ran around the corner and the lazy dog followed. The fight began. The quick brown fox, being quick and agile, dodged and weaved around the lazy dog's sloppy strikes. The lazy dog, with its powerful jaws, tried to sink its teeth into the quick brown fox's fast-moving body. The quick brown fox snapped back at the lazy dog with a sharp... Read More\n2-"
  }
]

3) returns

[
  [
    {
      "generated_text": "The quick brown fox outcropping her was an easy and natural fit. She was so fast and agile even those with great balance would have had difficulty taking a step or two. For another of her big hindrances, I was able to pick"
    }
  ],
  [
    {
      "generated_text": "jumps over the lazy dog, but I can't help but notice she is wearing some sort of dress (which is a fitting comparison for this part of town), and her ears are covered with small, bright pink earrings (which are quite fitting"
    }
  ]
]