Export sampling params and update llama_processor

netdur / llama_cpp_dart

dart binding for llama.cpp

MIT License

169 stars 22 forks source link

Export sampling params and update llama_processor #11

Closed danemadsen closed 10 months ago

danemadsen commented 10 months ago

Added support for SamplingParams to LlamaProcessor.

Also added SamplingProcessor to the Llama constructor instead of passing it as a param to Llama.getNext(). You can change this back if you like but it makes prompting with the LlamaProcessor cleaner as this way SamplingParam doesn't need to be passed to the isolate for each prompt.

netdur commented 10 months ago

@danemadsen I have just committed sampling code, seems it made outputs follows system instruction more closely... can you play with it and see if that works for you? I will accept your changes.