Lightning-AI / lit-llama

Implementation of the LLaMA language model based on nanoGPT. Supports flash attention, Int8 and GPTQ 4bit quantization, LoRA and LLaMA-Adapter fine-tuning, pre-training. Apache 2.0-licensed.
Apache License 2.0
6k stars 520 forks source link

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 #467

Open AI-Zebra opened 1 year ago

AI-Zebra commented 1 year ago

I have fine tuned a llama 7b model and when I try to run the fine tuned model I get below error:

python generate/lora.py 16-true Loading model ... Time to load model: 8.74 seconds. Traceback (most recent call last): File "/home/azureuser/lit-llama/generate/lora.py", line 129, in CLI(main) File "/usr/local/lib/python3.10/dist-packages/jsonargparse/_cli.py", line 96, in CLI return _run_component(components, cfg_init) File "/usr/local/lib/python3.10/dist-packages/jsonargparse/_cli.py", line 181, in _run_component return component(*cfg) File "/home/azureuser/lit-llama/generate/lora.py", line 101, in main output = generate( File "/usr/local/lib/python3.10/dist-packages/torch/utils/_contextlib.py", line 115, in decorate_context return func(args, **kwargs) File "/home/azureuser/lit-llama/generate.py", line 74, in generate idx_next = torch.multinomial(probs, num_samples=1).to(dtype=dtype) RuntimeError: probability tensor contains either inf, nan or element < 0