Consider using huggingface_hub library for Inference

[Wauplin]

In awesome_chat.py, you implement the inference method huggingface_model_inference (that uses the Hugging Face Hub under the hood) using requests and custom code. One thing you can do to deduplicate some code is to use huggingface_hub library and in particular its InferenceAPI class. You can find a guide here on how to use it.

For example, you can replace:

if task == "text-to-image":
    text = data["text"]
    response = requests.post(task_url, headers=HUGGINGFACE_HEADERS, json={"inputs": text})
    img_data = response.content
    img = Image.open(BytesIO(img_data))

by

if task == "text-to-image":
    img = InferenceApi(repo_id=model_id, task=task, token=token)(inputs=data["text"])

Some advantages of using huggingface_hub are:

• it checks if model and task are compatible
• if the user is logged in using huggingface-cli login, the token is automatically retrieved
• (limited) some data parsing when possible (image => PIL)

Disclaimer: I am a maintainer of huggingface_hub. If you need any help with the integration, I'd be happy to help! :smiley:

[tricktreat]

@Wauplin Thanks for your practical suggestion, we'll incorporate it into the plan!

[tricktreat]

Just to be sure, not all tasks huggingface_hub are supported? @Wauplin

[Wauplin]

@tricktreat all tasks that the Inference API on the Hub support are supported by this client (it's "just" a wrapper around requests). However only a few tasks will have its output parsed correctly. Otherwise you can get the raw response from the server (see reference).

If you see have feedback on the InferenceAPI client itself, please let me know. I'd glad to improve it if needed :)

[tricktreat]

@Wauplin Thanks. I simply tried the image-to-image task and got {'error': 'Task image-to-image is invalid'}. I think maybe not all tasks are supported.

import io
from diffusers.utils import load_image
from huggingface_hub.inference_api import InferenceApi

def image_to_bytes(img_url):
    img_byte = io.BytesIO()
    load_image(img_url).save(img_byte, format="jpeg")
    img_data = img_byte.getvalue()
    return img_data

inference = InferenceApi("lambdalabs/sd-image-variations-diffusers", token="****")
result = inference(data=image_to_bytes("https://raw.githubusercontent.com/justinpinkney/stable-diffusion/main/assets/im-vars-thin.jpg"))
print(result)

I also checked the code and here seems to be the currently supported tasks.

[Wauplin]

Oh right, thanks for noticing! This list is out of date then. I created an issue on huggingface_hub (https://github.com/huggingface/huggingface_hub/issues/1424)

:warning: You copy-pasted your token in the snippet above. I advice you to revoke it right now on https://huggingface.co/settings/tokens. For next tests you can use huggingface-cli login to login your machine so that you don't need to paste it in plain in your scripts.

[tricktreat]

Thanks for the reminder. I've edited it.

[Wauplin]

@tricktreat You really need to revoke it, i.e. create a new one. You can do that in your settings by clicking on Manage > Invalidate and refresh. You'll need to update it in the applications you have but it'll be safer this way.

On Github even when you edit a comment the previous version of the text can be accessed (if you see the little arrow on the right of edited in your comment). So your token is still visible to everyone at the moment :confused:

[tricktreat]

@Wauplin Thanks again! I was negligent about this. I have now completed the token reset according to your instructions.