If the user gives too long a prompt the GPU will throw a CUDA out of memory error and the whole PC/server requires rebooting. Does anyone have any suggestions for how to limit the length of the user input to prevent this?
I believe the easiest way to do this would be to add a character limit to the input box on the UI but that wouldn't cover the CLI option, and I can't find info on how to do this.
If the user gives too long a prompt the GPU will throw a CUDA out of memory error and the whole PC/server requires rebooting. Does anyone have any suggestions for how to limit the length of the user input to prevent this? I believe the easiest way to do this would be to add a character limit to the input box on the UI but that wouldn't cover the CLI option, and I can't find info on how to do this.