Error when querying hugging face model

poe-platform / server-bot-quick-start

Tutorial for Poe server bots

1.31k stars 192 forks source link

Error when querying hugging face model #56

Closed Shivak11 closed 8 months ago

Shivak11 commented 10 months ago

I tried the huggingface bot script. I am encountering a 404 error. Looks likes MODAL is expecting the model to be run locally. Is that correct?

anmolsingh95 commented 8 months ago

I just tried the huggingface script and it seems or be working for me. We are the AsyncInferenceClient from the huggingface_hub library. According to the comments in the code:

The model to run inference with. Can be a model id hosted on the Hugging Face Hub, e.g. bigcode/starcoder or a URL to a deployed Inference Endpoint. Defaults to None, in which case a recommended model is automatically selected for the task.