RuntimeError: probability tensor contains either `inf`, `nan` or element < 0

Describe the bug

Traceback (most recent call last): File "F:\gptai\text-generation-webui-snapshot-2024-04-28\modules\callbacks.py", line 61, in gentask ret = self.mfunc(callback=_callback, args, self.kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\gptai\text-generation-webui-snapshot-2024-04-28\modules\text_generation.py", line 376, in generate_with_callback shared.model.generate(kwargs) File "F:\gptai\text-generation-webui-snapshot-2024-04-28\installer_files\env\Lib\site-packages\awq\models\base.py", line 110, in generate return self.model.generate(args, *kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\gptai\text-generation-webui-snapshot-2024-04-28\installer_files\env\Lib\site-packages\torch\utils_contextlib.py", line 115, in decorate_context return func(args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^ File "F:\gptai\text-generation-webui-snapshot-2024-04-28\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 1559, in generate result = self._assisted_decoding( ^^^^^^^^^^^^^^^^^^^^^^^^ File "F:\gptai\text-generation-webui-snapshot-2024-04-28\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 4718, in _assisted_decoding selected_tokens = torch.multinomial(probs[0, :, :], num_samples=1).squeeze(1)[None, :] ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ RuntimeError: probability tensor contains either inf, nan or element < 0

Is there an existing issue for this?

[X] I have searched the existing issues

Reproduction

just type anything in the webui

Screenshot

No response

Logs

03:02:50-499440 INFO     "Midnight Enigma" preset:
{   'temperature': 0.98,
    'top_p': 0.37,
    'top_k': 100,
    'repetition_penalty': 1.18}
Traceback (most recent call last):
  File "F:\gptai\text-generation-webui-snapshot-2024-04-28\modules\callbacks.py", line 61, in gentask
    ret = self.mfunc(callback=_callback, *args, **self.kwargs)
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\gptai\text-generation-webui-snapshot-2024-04-28\modules\text_generation.py", line 376, in generate_with_callback
    shared.model.generate(**kwargs)
  File "F:\gptai\text-generation-webui-snapshot-2024-04-28\installer_files\env\Lib\site-packages\awq\models\base.py", line 110, in generate
    return self.model.generate(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\gptai\text-generation-webui-snapshot-2024-04-28\installer_files\env\Lib\site-packages\torch\utils\_contextlib.py", line 115, in decorate_context
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "F:\gptai\text-generation-webui-snapshot-2024-04-28\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 1559, in generate
    result = self._assisted_decoding(
             ^^^^^^^^^^^^^^^^^^^^^^^^
  File "F:\gptai\text-generation-webui-snapshot-2024-04-28\installer_files\env\Lib\site-packages\transformers\generation\utils.py", line 4718, in _assisted_decoding
    selected_tokens = torch.multinomial(probs[0, :, :], num_samples=1).squeeze(1)[None, :]
                      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0
Output generated in 12.25 seconds (0.16 tokens/s, 2 tokens, context 3369, seed 238601171)

System Info

CPU: AMD Ryzen 5 5600X 6-Core Processor 3.70 GHz
GPU: Nvidia GeForce 3060 12gb

oobabooga / text-generation-webui