Download Petals models from Torrent

[filopedraz]

Description

Instead of downloading the models from HF, the services should fetch the weights from Torrent.

Dependencies

• This implementation needs to be working and hosted on a server.

[biswaroop1547]

I can see few directions to achieve this (with questions):

• Serve a specific model as service that'll be downloaded from torrent only (e.g Codellama34B/StableBeluga2/something else?) that already has torrent and seeding (by us) through from-hf-to-torrent.
  • do it with a smaller model (codellama-7b?) given 34B would take too much space and compute for users to use.
• Work on making all model services torrent compatible (this might be blocked currently given there are some resource constraints on Hetzner VM)

Now eitherway to download using torrent, a bittorrent client will be required, given we are going with transmission client (might change later), it'd be a dependency for each service based on our current structure. We would have to use pre-built docker images of transmission and start them up in background like this, or merge transmission's dockerfile contents with our current service dockerfiles -- but that'd be too bloated I think.

What are your opinions here for the above two things? @casperdcl @filopedraz also since we are working towards removing usage of docker, how things should look like here?

[filopedraz]

Petals doesn't download the entire model, but only the layers that can be consumed by the client. We need to check how they handle that and then "replicate" the behavior to use torrent instead of HF.