The original binding.cpp only applies RNG seed in ModelOptions, custom seeds passed by PredictOptions for individual prediction requests are not used. The example script is modified to illustrate its use.
Could be a fix to https://github.com/go-skynet/LocalAI/issues/676, because LocalAI doesn't set a model-level seed, so the default value 0 is used, which prevents llama.cpp from using random RNG seeds.
Some other changes:
A few useless RNGs in binding.cpp are removed;
RoPE default values for prediction are set. (It's a bit weird to patch it in the frontend like LocalAI IMO)
P.S. I've only made the minimum changes, but I think we can use prediction-level seeds only and remove model-level seeds, like llama.cpp's official server example. For me it doesn't make sense to stick to a seed for a long-running backend service.
The original
binding.cpponly applies RNG seed in ModelOptions, custom seeds passed by PredictOptions for individual prediction requests are not used. The example script is modified to illustrate its use.Could be a fix to https://github.com/go-skynet/LocalAI/issues/676, because LocalAI doesn't set a model-level seed, so the default value 0 is used, which prevents llama.cpp from using random RNG seeds.
Some other changes:
binding.cppare removed;P.S. I've only made the minimum changes, but I think we can use prediction-level seeds only and remove model-level seeds, like llama.cpp's official server example. For me it doesn't make sense to stick to a seed for a long-running backend service.