henk717
commented
5 months ago Your GPU ran out of memory. If you need a more efficient alternative check out https://koboldai.org/cpp or run the model on colab https://koboldai.org/colabcpp
If you would like one on one help visit us at https://koboldai.org/discord
File "aiserver.py", line 4937, in generate genout, already_generated = tpool.execute(_generate, txt, minimum, maximum, found_entries) File "B:\python\lib\site-packages\eventlet\tpool.py", line 132, in execute six.reraise(c, e, tb) File "B:\python\lib\site-packages\six.py", line 719, in reraise raise value File "B:\python\lib\site-packages\eventlet\tpool.py", line 86, in tworker rv = meth(*args, *kwargs) File "aiserver.py", line 4860, in _generate genout = generator( File "B:\python\lib\site-packages\torch\autograd\grad_mode.py", line 27, in decorate_context return func(args, **kwargs) File "B:\python\lib\site-packages\transformers\generation_utils.py", line 1324, in generate model_kwargs["attention_mask"] = self._prepare_attention_mask_for_generation( File "B:\python\lib\site-packages\transformers\generation_utils.py", line 556, in _prepare_attention_mask_for_generation is_pad_token_in_inputs = (pad_token_id is not None) and (pad_token_id in inputs) File "B:\python\lib\site-packages\torch_tensor.py", line 757, in contains return (element == self).any().item() # type: ignore[union-attr] RuntimeError: CUDA error: out of memory CUDA kernel errors might be asynchronously reported at some other API call,so the stacktrace below might be incorrect. For debugging consider passing CUDA_LAUNCH_BLOCKING=1.