search

rendezqueue / rendezllama

CLI for llama.cpp with various commands to guide, edit, and regenerate tokens on the fly.
ISC License
11 stars 1 forks source link

feat(option): Custom sampling order #36

Open grencez opened 10 months ago

grencez commented 10 months ago

working idea:


(language
 ((infer_via sampling)
  (adjust_thru (())
   (xtc (probability 0.5) (threshold 0.1))
  )
  ((pick_via mirostat) (tau 5.0) (eta 0.1))
 )
)
grencez commented 4 months ago

I'm going to wait for llama.cpp's sampling refactor. It looks like some of the project's CLI is going to be exposed as a library, so the most reliable implementation on our end could just hook into that.