search

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
https://petals.dev
MIT License
8.89k stars 489 forks source link

mitigate unnecessary swarm rebalancing #493

Open iateadonut opened 10 months ago

iateadonut commented 10 months ago

https://github.com/bigscience-workshop/petals/issues/389

This works by trying 'should_choose_other_blocks' twice more over 2-3 minutes if it returns a 'true' value.