search

nomic-ai / gpt4all

GPT4All: Run Local LLMs on Any Device. Open-source and available for commercial use.
https://nomic.ai/gpt4all
MIT License
69.15k stars 7.59k forks source link

[Feature] Please Add Option for Llama 3 70B Parameters #2334

Open yoyokits opened 4 months ago

yoyokits commented 4 months ago

Feature Request

In the model download window, there is no option to select Llama 3 70B, please add it.

woheller69 commented 4 months ago

you can simply manually copy it to the model folder

yoyokits commented 4 months ago

I didn't know that, thank you. I will add it manually.

woheller69 commented 4 months ago

Make sure to take care of this: https://github.com/nomic-ai/gpt4all/issues/2239

Bresenham commented 1 month ago

Do you mind giving instructions how to add Llama 3.1 70B manually? I am on windows. I edited the models.json file by adding an additional dictionary that links to a Llama 3.1 70B from huggingface. However, gpt4all.exe doesn't seem to take the changed models.json into account - the "models" section in the GUI didn't change. I am now also downloading Meta-Llama-3.1-70B-Instruct directly from Meta but I am unsure how I could integrate it into GPT4All.

tek911 commented 3 weeks ago

Hey, i think the limiting factor here is to pull the model hugging face has a gating where you have to provide your contact info to the meta team (some models require it). I dont know if 40B or 70B can be out of the box provided by gpt4all as a download. That said, sounds like an enhancement request to the project to support HF models that gate with asking for additional info fields. But then you gotta figure out how to check back and see the status and such, may be more involved than i think project goals will permit but im not on the project so i have no clue.