search

google / gemma_pytorch

The official PyTorch implementation of Google's Gemma models
https://ai.google.dev/gemma
Apache License 2.0
5.26k stars 503 forks source link

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 #13

Closed 2579356425 closed 7 months ago

2579356425 commented 7 months ago

I get error RuntimeError: probability tensor contains eitherinf,nanor element < 0 when infering with gemma-7b-quant.ckpt. The environments are as follows:

fairscale==0.4.13 filelock==3.9.0 fsspec==2023.4.0 gemma==0.1 immutabledict==4.1.0 Jinja2==3.1.2 MarkupSafe==2.1.3 mpmath==1.3.0 networkx==3.2.1 numpy==1.24.4 nvidia-cublas-cu11==11.11.3.6 nvidia-cuda-cupti-cu11==11.8.87 nvidia-cuda-nvrtc-cu11==11.8.89 nvidia-cuda-runtime-cu11==11.8.89 nvidia-cudnn-cu11==8.7.0.84 nvidia-cufft-cu11==10.9.0.58 nvidia-curand-cu11==10.3.0.86 nvidia-cusolver-cu11==11.4.1.48 nvidia-cusparse-cu11==11.7.5.86 nvidia-nccl-cu11==2.19.3 nvidia-nvtx-cu11==11.8.86 pillow==10.2.0 sentencepiece==0.1.99 sympy==1.12 torch==2.2.1+cu118 torchaudio==2.2.1+cu118 torchvision==0.17.1+cu118 triton==2.2.0 typing_extensions==4.8.0

chenhaijian520 commented 7 months ago

me too， did you solve it?

2579356425 commented 7 months ago

me too， did you solve it?

I set model_config.quant = True in run.py. The problem is solved. But the answer is wrong.

chenhaijian520 commented 7 months ago

it works ! but it’s answer is very rude ~.~

michaelmoynihan commented 7 months ago

Duplicate issue #10.

xiangru2020 commented 6 months ago

The same problem i meet. anybody who knows why?

xingranzh commented 3 months ago

I also meet the same problem. It seems that this bug has not been solved after nearly half a year.