search

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
https://petals.dev
MIT License
8.89k stars 490 forks source link

Reachability Issue for private swarm #551

Closed VarunJoshi10 closed 4 months ago

VarunJoshi10 commented 5 months ago

I am creating a private swarm using backbone peer hosted at aws EC2 instance and trying to connect my machine's GPU but getting reachability issues. I have followed the list of commands listed below. python3 -m petals.cli.run_dht --host_maddrs /ip4/0.0.0.0/tcp/31337 --identity_path bootstrap1.id

I am getting reachability as False on backbone peer: reachability.rpc_check(remote_peer=...YELDdS, check_peer=...YELDdS) -> False

I have followed commands from petals documentation for creating backbone peer and running petals server python3 -m petals.cli.run_server meta-llama/Llama-2-13b-hf --token mytoken --initial_peers /ip4/EC2_public_ip/tcp/31337/p2p/Qm_mybackbone_peer

adnanbaqi commented 4 months ago

Brother use , docker image for health monitor the issue won't happen