search

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
https://petals.dev
MIT License
8.89k stars 490 forks source link

Fix p2p pushing in rpc_inference, support transformers 4.38.2 #563

Closed justheuristic closed 3 months ago

justheuristic commented 3 months ago

This pull request solves https://github.com/bigscience-workshop/petals/pull/560 using a solution proposed by @miaoqijun