jeromeku
commented
1 year ago Did some additional debugging.
Something odd is going on during the call to ppo_trainer.generate.
Tracing this call ultimately leads to this line of the underlying LlamaModel, which tries to embed the input_ids.
I checked that the input_ids are all within the min and max embedding dimensions (= 32000, i.e., the vocab size). Setting a breakpoint here and passing in each input_id individually, the first embedding call works, but subsequent calls result in the CUDA error per the previous post (CUDA device-side assert error). Also the order in which the inputs are embedded doesn't seem to matter -- the CUDA error is triggered for subsequent embedding calls after the first embedding call.
younesbelkada
Trying to run stack-llama rl_training script with the following reward model and this SFT model.
The SFT model was created from a base Llama 7B model (weights converted using llama convert script) merged with SFT adapter weights using the merge script.
Tokenizer is that from the converted Llama 7B model.
Running the
rl_training.pyscript with default parameters and with reward model, SFT model, and tokenizer per above usingaccelerate launchresults in a CUDA device-side assertion error:Notes:
supervised_fine_tuning.pyandreward_modeling.pyscripts and was able to run both successfully.rl_training.pyscript but always get the above error.Here is my environment:
Device is:
NVIDIA A100-SXM4-80GBAny ideas what might be causing this?