Add plumming for additional model parameters.

simonw / llm-llama-cpp

LLM plugin for running models using llama.cpp

Apache License 2.0

136 stars 19 forks source link

Add plumming for additional model parameters. #21

Open hhughes opened 11 months ago

hhughes commented 11 months ago

Defaults set to current values in llama-cpp-python. Descriptions taken from llama-cpp-python documentation where defined.

New parameters:

suffix
temperature
top_p
logprobs
echo
stop
frequency_penalty
presence_penalty
repeat_penalty
top_k
stream
tfs_z
mirostat_mode
mirostat_tau
mirostat_eta
model

hhughes commented 11 months ago

Motivation: I wanted to be able to tune temperature, top_k and top_p when using llama via llm package. I copied almost all current params wholesale, if you think some do not belong I'm happy to adjust the CL.

I left out stopping_criteria, logits_processor and grammar because pydantic didn't like their types (and I don't need them for now).

aawadat commented 2 months ago