KoboldAI / KoboldAI-Client

For GGUF support, see KoboldCPP: https://github.com/LostRuins/koboldcpp
https://koboldai.com
GNU Affero General Public License v3.0
3.51k stars 758 forks source link

Output Length and Context Tokens can be set to 512, resulting in error #298

Open biscober opened 1 year ago

biscober commented 1 year ago

Ideally, users should not be able to set a selection that will fail on inference and KoboldAI will sanitize user input to prevent such a situation. This is similar to #284, where my primary concern is that a malicious user on the KoboldAI Horde could submit jobs that cannot be processed by a horde worker, which will then cause the worker to 'drop' the jobs repeatedly until it's knocked into maintenance mode.