search

meta-llama / llama-models

Utilities intended for use with Llama models.
Other
4.88k stars 838 forks source link

Run Llama on CPU #207

Closed naveenmaan closed 1 week ago

naveenmaan commented 2 weeks ago

Hello, after downloading the 3.2 1B model. I am trying to run the model on CPU but getting various error that I do not have GPU or cuda is missing. I tried to change the code and all but still facing the issue.

My task is to run the 1B model on my CPU.

Can anyone help me?

varunfb commented 1 week ago

Have you tried using ollama? This should simplify a lot of things for you to run it on CPU.

naveenmaan commented 1 week ago

I want a direct approach instead of third-party lib.

ashwinb commented 1 week ago

This is not the intended use of this specific repository. The goal of llama-models is to show the architecture in its simplest form. Inference solutions can get extremely complicated because there's a multitude of environments. There are various high quality implementations and Ollama is one of the best in our opinion for running on CPU.