RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 - Githubissues

huggingface / trl

Train transformer language models with reinforcement learning.

http://hf.co/docs/trl

Apache License 2.0

10.14k stars 1.28k forks source link

RuntimeError: probability tensor contains either `inf`, `nan` or element < 0 #2205

Open himanshushukla12 opened 1 month ago

himanshushukla12 commented 1 month ago

System Info

System Info

Environment Details:

trl version: 0.11.1
transformers version: 4.45.1
Python version: 3.10.11
Operating System: Linux 4649c3747948 6.8.0-41-generic #41-Ubuntu SMP PREEMPT_DYNAMIC Fri Aug 2 20:41:06 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
CUDA version: 12.1 (V12.1.105)

Information

[X] The official example scripts
[ ] My own modified scripts

Tasks

[X] An officially supported task in the examples folder
[ ] My own task or dataset (give details below)

: what is AI? : : what is AI? : ?',?', ?

himanshushukla12 commented 1 month ago

Issue occurs when we have multi-GPU in our system. If we use CUDA_VISIBLE_DEVICE=0 it works fine