Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
Apache License 2.0
6k
stars
520
forks
source link
RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 #467
I have fine tuned a llama 7b model and when I try to run the fine tuned model I get below error:
python generate/lora.py
16-true
Loading model ...
Time to load model: 8.74 seconds.
Traceback (most recent call last):
File "/home/azureuser/lit-llama/generate/lora.py", line 129, in
CLI(main)
File "/usr/local/lib/python3.10/dist-packages/jsonargparse/_cli.py", line 96, in CLI
return _run_component(components, cfg_init)
File "/usr/local/lib/python3.10/dist-packages/jsonargparse/_cli.py", line 181, in _run_component
return component(*cfg)
File "/home/azureuser/lit-llama/generate/lora.py", line 101, in main
output = generate(
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(args, **kwargs)
File "/home/azureuser/lit-llama/generate.py", line 74, in generate
idx_next = torch.multinomial(probs, num_samples=1).to(dtype=dtype)
RuntimeError: probability tensor contains either inf, nan or element < 0
I have fine tuned a llama 7b model and when I try to run the fine tuned model I get below error:
python generate/lora.py 16-true Loading model ... Time to load model: 8.74 seconds. Traceback (most recent call last): File "/home/azureuser/lit-llama/generate/lora.py", line 129, in
CLI(main)
File "/usr/local/lib/python3.10/dist-packages/jsonargparse/_cli.py", line 96, in CLI
return _run_component(components, cfg_init)
File "/usr/local/lib/python3.10/dist-packages/jsonargparse/_cli.py", line 181, in _run_component
return component(*cfg)
File "/home/azureuser/lit-llama/generate/lora.py", line 101, in main
output = generate(
File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context
return func(args, **kwargs)
File "/home/azureuser/lit-llama/generate.py", line 74, in generate
idx_next = torch.multinomial(probs, num_samples=1).to(dtype=dtype)
RuntimeError: probability tensor contains either
inf
,nan
or element < 0