search

tloen / llama-int8

Quantized inference code for LLaMA models
GNU General Public License v3.0
1.05k stars 105 forks source link

Producing nan Tensors #20

Open Bryan-Lavender opened 1 year ago

Bryan-Lavender commented 1 year ago

generate.py sometimes produces tensors with nan and sometimes does not and I cannot see any support for when this happens. I am using the given example.