search

bigscience-workshop / petals

🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading
https://petals.dev
MIT License
9.27k stars 524 forks source link

Feature Request: Distributed inference API where the "miners" are paid. #556

Open NewtonTrendy opened 9 months ago

NewtonTrendy commented 9 months ago

I'm imagining a situation where people can infer for others and make a small profit. In practicality it is similar to, but more useful than, cryptocurrency mining. You would get paid a small amount per percent of a request you inferred by users of the inference API. If the system was built people could load their own models into the cluster and it would make sure demand is met at market rate. This would improve access for individuals to large compute in projects they want to develop, and would not be a worse use of compute than cryptocurrency mining.

Priyanshupareek commented 8 months ago

Hey @NewtonTrendy I was thinking on similar lines and am working on this. Wanna chat more on discord?

mrseeker commented 5 months ago

Hi all,

There is one already: https://github.com/hypertensor-blockchain/petals_tensor