search

deep-diver / llamaduo

This project showcases an LLMOps pipeline that fine-tunes a small-size LLM model to prepare for the outage of the service LLM.
https://huggingface.co/papers/2408.13467
Apache License 2.0
286 stars 29 forks source link

Does the local_lm works with cpu only? #27

Closed pacozaa closed 1 month ago

pacozaa commented 1 month ago

Hi interesting project! I would like to know if this works with cpu only machine like mac.

juyongjiang commented 1 month ago

Hi interesting project! I would like to know if this works with cpu only machine like mac.

@pacozaa Hi, thank you for your interest. I believe it can operate on CPU-only machines, but it will not be as fast as on GPU devices. Please refer to the guideline at https://rentry.org/cpu-lora and use https://github.com/ggerganov/llama.cpp. : )