chat.petals integration with Hugging Face Spaces

[justheuristic]

Currently, you can use the model either directly, via this notebook, or through http://chat.petals.ml/ .

The latter is a self-hosted chat app that runs code similar to the colab notebook, but wraps it with a simple web server. The source code is available here.

It would be great if anybody could fork the chat project and reuse it for their pet projects. Thing is, not everyone has a server to host that web server - and it is not always convenient to share jupyter notebooks for everything.

To make it easier to create similar apps, we can create an example with Spaces. Hugging Face Spaces allows you to run simple web apps at the expense of HF.

References:

• @younesbelkada has made a space with the early version of Petals: https://huggingface.co/spaces/ybelkada/petals/tree/main
• There is another related project that integrates petals 1.0.0, https://huggingface.co/spaces/hivemind-personalized-chat/chat-gradio/tree/main

Task: create something similar to chat.petals on top of HF spaces. One way to do this is to fork @younesbelkada 's project and update it to the latest stable version of Petals

[gururise]

I tried here: https://huggingface.co/spaces/yahma/BLOOMZ_Compare

The gradio app I wrote works standalone on my machine, but fails on HuggingFace SPACES with:

hivemind.p2p.p2p_daemon_bindings.utils.P2PDaemonError: Daemon failed to start: 2023/02/04 00:09:03 failed to connect to bootstrap peers

I think its a DHT issue related to SPACES. Not sure if there are ports that need to be opened for DHT to work properly?

[justheuristic]

Curious. Last year, HF spaces instances would not connect to bootstrap peers with any ports other than 80 or 433 (iirc), but i thought they removed that filter a while ago. I'll check with @younesbelkada (the author of the original space) perhaps he knows more .

[borzunov]

For the record: it is easy to set up a new bootstrap peer on port 80 (accessible from HF Spaces) but it won't help because Petals servers ran by users listen to lots of different ports (so the client will connect to bootstrap peers but won't be able to connect to them). So let's wait for the @younesbelkada's response.

@gururise, thanks for making the HF Space! Hopefully, we'll make it working soon.

[borzunov]

See also this space made by @slush0: https://huggingface.co/spaces/slush0/petals-bloom-playground

It uses the websocket API http://chat.petals.ml/api/v2 instead of running the Petals client directly.