search

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
https://petals.dev
MIT License
9.27k stars 524 forks source link

Fix p2p pushing in rpc_inference, support transformers 4.38.2 #563

Closed justheuristic closed 8 months ago

justheuristic commented 8 months ago

This pull request solves https://github.com/bigscience-workshop/petals/pull/560 using a solution proposed by @miaoqijun