search

turboderp / exllamav2

A fast inference library for running LLMs locally on modern consumer-class GPUs
MIT License
3.2k stars 236 forks source link

what is `path_to_model` in the starter command in the readme ? #318

Closed hemangjoshi37a closed 5 months ago

hemangjoshi37a commented 5 months ago

I am little bit new here so dont know what does it mean by path_to_model here . where to download this model files ?

turboderp commented 5 months ago

You can download them from Hugging Face. For instance here, simple way is to click each of the arrows and stick all the downloaded files in one folder. Then use the path to that folder.

There's also git clone if you have git-lfs installed, or alternatively there's huggingface-cli.