October binary update

[martindevans]

Updated binaries to llama.cpp c35e586ea57221844442c65a1172498c54971cb0, built with https://github.com/SciSharp/LLamaSharp/actions/runs/10984319249

This is an unusually large change due to a redesign of the llama.cpp sampling API. The new sampling system has all been exposed in a safe way except for custom samplers, which are not implemented and will need to be tackled in a future update.

Assorted notable changes:

• Mirostat and Mirostat2 pipeline classes have been removed (they can easily be re-implemented if necessary)
• Removed all of the IInferenceParams sampling parameters, this must be done through a pipeline
• Sampling parameters in WebUI are commented out (removal of IInferenceParams properties broke it)
• Guidance is not supported by llama.cpp, deleted the guidance example
• Removed our entire GBNF grammar parser, that's now part of llama.cpp
• Sampling no longer needs to allocate a large temporary array for every token sampled 🥳

Testing:

• [x] Windows CPU
• [x] Windows CUDA
• [x] Windows Vulkan
• [x] Linux CPU
• [x] Linux CUDA
• [x] Linux Vulkan
• [x] MacOS CPU
• [x] MacOS Metal

[m0nsky]

Test application is running fine on:

• Windows Vulkan
• Linux CPU
• Linux CUDA
• Linux Vulkan

I noticed seed was removed from the ModelParams, how can we set the seed after this PR?

Edit I see there's a SetSeed() in LLamaContext. I tried using context.SetSeed(1337); but it throws System.AggregateException: One or more errors occurred. (Unable to find an entry point named 'llama_set_rng_seed' in DLL 'llama'.)

[martindevans]

I noticed seed was removed from the ModelParams, how can we set the seed after this PR?

The seed is now set per-sampler, see my other comment for an example.

I'm glad you mentioned this though, because I didn't pass that through to the higher level! I'll add a Seed property to DefaultSamplingPipeline, which is the more direct replacement for ModelParams.

there's a SetSeed() in LLamaContext

Good catch, that method should have been removed too.

[m0nsky]

Can confirm the Seed in the DefaultSamplingPipeline now works as expected!

[SignalRT]

Test is running fine on:

• MacOS ARM
• MacOS Intel/CPU