To support Hugging Face model

b4rtaz / distributed-llama

Tensor parallelism is all you need. Run LLMs on weak devices or make powerful devices even more powerful by distributing the workload and dividing the RAM usage.

MIT License

1.03k stars 69 forks source link

To support Hugging Face model #30

Closed hyperbolic-c closed 1 month ago

hyperbolic-c commented 2 months ago

When download the model from Meta website using email url, there are often get network problems 403 Forbidden. Is there any support for hugging face model possible ? Thanks !

b4rtaz commented 2 months ago

Hello @hyperbolic-c,

I uploaded Llama 3 8B Q40 here: https://huggingface.co/b4rtaz/llama-3-8b-distributed-llama

hyperbolic-c commented 2 months ago

Hello @hyperbolic-c,

I uploaded Llama 3 8B Q40 here: https://huggingface.co/b4rtaz/llama-3-8b-distributed-llama

Yeah ! Thanks a lot for your contribution ! !

hyperbolic-c commented 1 month ago

@b4rtaz Hi, do the model files need to be reconverted with version updates? If it could also support convert from hugging face format not only .pth format ? Thanks !!

hyperbolic-c commented 1 month ago

Hey, the report.pdf you mentioned at this comment was missing, could you please offer some details ? Thank you ! @b4rtaz

b4rtaz commented 1 month ago

@hyperbolic-c please check here.

hyperbolic-c commented 1 month ago

@b4rtaz Sorry, I can not open it, "Error rendering embedded code Invalid PDF". After download it also can not open.

b4rtaz commented 1 month ago

What browser are you using? I've just checked it and it works on Firefox/Chrome/Safari.

hyperbolic-c commented 1 month ago

I have checked it on edge and chrome, none of them work.

DifferentialityDevelopment commented 1 month ago

I was able to open it up just fine as well

hyperbolic-c commented 1 month ago

Thanks! I have open it on my phone. It can not open in the computer browser. That's weird.