search

ggerganov / llama.cpp

LLM inference in C/C++
MIT License
64.69k stars 9.27k forks source link

Run Llama.cpp in silent mode #8507

Closed MathiasSchindler closed 1 week ago

MathiasSchindler commented 1 month ago

Discussed in https://github.com/ggerganov/llama.cpp/discussions/777

Originally posted by **OsaCode** April 5, 2023 Is there an option to run the executable using a saved prompt without printing anything but the prediction by the selected model ? Some kind of silent mode in opposition to the --verbose.

I believe this is a good first issue to introduce a --silent flag.

AmgadHasan commented 1 month ago

Running it with --log-disable --no-display-prompt flags does what you want

 $ ~/llama.cpp/b3405/bin/llama-cli -m ./models/Lite-Mistral-150M-v2-Instruct-Q8_0.gguf -p "<s>system
You are a helpful assistant</s>
<s>user
Write an article about Deep Learning. The article should be 1000 words at least</s> 
<s>assistant
" -n 1024 --log-disable --no-display-prompt

Deep learning has revolutionized the way we learn and use artificial intelligence. It has revolutionized many industries, including healthcare, finance, and manufacturing. In this article, we'll explore the fascinating process behind deep learning, covering its various applications, and discuss its potential as a powerful tool for enhancing human intelligence.
github-actions[bot] commented 1 week ago

This issue was closed because it has been inactive for 14 days since being marked as stale.