Closed tontan1998 closed 1 year ago
Here are the libraries:
Looks like model collapsed. Have you tried changing the LR for the larger model? Also training logs would be useful. Maybe @edbeeching knows good setting for the larger models.
@tontan1998 did you solve it? I also get this error
Hi @akk-123 @tontan1998
I did not managed to reproduce the issue with edbeeching/gpt-neo-1.3B-imdb
Can you try with the most up to date script as it has changed a bit
Also I will share with you the output of pip freeze
accelerate==0.18.0
torch==2.0.0
transformers==4.28.0.dev0
peft==0.3.0.dev0
Closing the issue for now, feel free to re-open if there's an update.
accelerate==0.18.0 torch==2.0.0 transformers==4.28.0.dev0 peft==0.3.0.dev0
I meet the same problem in a docker image environment with this version of packages. The bug has gone after changing to a local machine. The code, model, and package version are all the same. But in the docker image, the code will get the bug.
i have smae issue, i get a same error when i run.
I have smae issue, using glm with trl. but not in trl, it works well.
I found that the model weights have gone to NaN after some training steps, causing model(input_ids)
to output Nan.
This means that the gradient which are being added to the model weights have NaNs . I simply changed the default option of max_grad_norm=None
to max_grad_norm=1
for PPOConfig
My PPOConfig looks like
PPOConfig(batch_size=args.generator.batch_size, seed=args.seed, max_grad_norm=1)
I have encountered the same issue. My solution is to reduce learning rate, and disable the "score scaling" feature, the later uses std directly in the denominator which is very risky imo. We should add an epsilon there? @lvwerra @younesbelkada
Would be a viable option for me. What do you think @zfang?
Would be a viable option for me. What do you think @zfang?
@lvwerra Created https://github.com/huggingface/trl/pull/727
accelerate==0.18.0 torch==2.0.0 transformers==4.28.0.dev0 peft==0.3.0.dev0
I meet the same problem in a docker image environment with this version of packages. The bug has gone after changing to a local machine. The code, model, and package version are all the same. But in the docker image, the code will get the bug.
How did you solve in the docker environment
I have this error running Mistral-Nemo-2407 in two RTX 3090 24gb, with FastChat.
If I use only 1 gpu and cpu offloading, this problem does not come up, but I have OOMs with long requests.
Hello! I try to use gpt2-sentiment_peft.py and I was change the model from
edbeeching/gpt-neo-125M-imdb
toedbeeching/gpt-neo-1.3B-imdb
but It is not working.source code