search

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
https://petals.dev
MIT License
8.89k stars 489 forks source link

Create model index in DHT #491

Closed borzunov closed 10 months ago

borzunov commented 10 months ago

This PR creates an index of models hosted in the swarm - it is useful to know which custom models users run and display them at https://health.petals.dev as "not officially supported" models.

After this PR:

In [1]: import hivemind 

In [2]: from petals.constants import PUBLIC_INITIAL_PEERS                                                                  

In [3]: dht = hivemind.DHT(initial_peers=PUBLIC_INITIAL_PEERS, client_mode=True, start=True)

In [4]: dht.get("_petals.models")                                                                  
Out[4]: ValueWithExpiration(value={'llama-65b-hf': ValueWithExpiration(value={'num_blocks': 80, 'repository': 'https://huggingface.co/huggyllama/llama-65b'}, expiration_time=1693462442.2670343)}, expiration
_time=1693462442.2670343)