ZzZZCHS
commented
1 month ago The inputs_embeds passed to self.llama_model.generate is unlikely to be None. The problem may be caused by different versions of the transformers or peft packages. Could you verify that the versions of these two packages match those specified in the requirements?
WinterCodeForEverything
Thanks for your excellent work, I download the pretrained checkpoint(ckpt_01_3214.pth) and run the evaluation code, but get:
rank1: Traceback (most recent call last): rank1: File "/data/projects/15003900/Chat-3D-v2/tasks/train.py", line 447, in
rank1: File "/data/projects/15003900/Chat-3D-v2/tasks/train.py", line 391, in main rank1: global_step = train( rank1: File "/data/projects/15003900/Chat-3D-v2/tasks/train.py", line 133, in train rank1: val_metrics = evaluate_all(model, model_without_ddp, val_loaders, epoch, global_step, device, config) rank1: File "/data/projects/15003900/Chat-3D-v2/tasks/train.py", line 186, in evaluate_all rank1: new_val_scores = evaluate(model, val_loader, epoch, global_step, device, config) rank1: File "/data/projects/15003900/Chat-3D-v2/tasks/train.py", line 220, in evaluate rank1: pred = model(batch, is_eval=True) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl rank1: return self._call_impl(*args, *kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl rank1: return forward_call(args, kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1636, in forward rank1: else self._run_ddp_forward(*inputs, kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/parallel/distributed.py", line 1454, in _run_ddp_forward rank1: return self.module(*inputs, *kwargs) # type: ignoreindex: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl rank1: return self._call_impl(args, kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl rank1: return forward_call(args, kwargs) rank1: File "/data/projects/15003900/Chat-3D-v2/models/chat3d.py", line 458, in forward rank1: return self.evaluate(kwargs) rank1: File "/data/projects/15003900/Chat-3D-v2/models/chat3d.py", line 434, in evaluate rank1: outputs = self.llama_model.generate( rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/peft/peft_model.py", line 1491, in generate rank1: outputs = self.base_model.generate(args, kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context rank1: return func(args, kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/transformers/generation/utils.py", line 2028, in generate rank1: result = self._beam_search( rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/transformers/generation/utils.py", line 3187, in _beam_search rank1: outputs = self(model_inputs, return_dict=True) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl rank1: return self._call_impl(args, kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl rank1: return forward_call(*args, kwargs) rank1: File "/data/projects/15003900/Chat-3D-v2/models/modeling_llama.py", line 1188, in forward rank1: outputs = self.model( rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl rank1: return self._call_impl(*args, *kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl rank1: return forward_call(args, kwargs) rank1: File "/data/projects/15003900/Chat-3D-v2/models/modeling_llama.py", line 1069, in forward rank1: layer_outputs = decoder_layer( rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl rank1: return self._call_impl(*args, kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl rank1: return forward_call(*args, *kwargs) rank1: File "/data/projects/15003900/Chat-3D-v2/models/modeling_llama.py", line 797, in forward rank1: hidden_states, self_attn_weights, present_key_value = self.self_attn( rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl rank1: return self._call_impl(args, kwargs) rank1: File "/home/project/15003900/miniconda3/envs/py310_cuda117/lib/python3.10/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl rank1: return forward_call(*args, **kwargs) rank1: File "/data/projects/15003900/Chat-3D-v2/models/modeling_llama.py", line 509, in forward rank1: query_states, key_states = apply_rotary_pos_emb(query_states, key_states, cos, sin, position_ids) rank1: File "/data/projects/15003900/Chat-3D-v2/models/modeling_llama.py", line 232, in apply_rotary_pos_emb rank1: cos = cos[position_ids].unsqueeze(unsqueeze_dim) rank1: IndexError: index is out of bounds for dimension with size 0.
I'm try to debug and find that in LlamaForCausalLM forward function, inputs_embeds is None and input_ids is a null tensor: inputs_embeds: None input_ids: tensor([], device='cuda:0', size=(5, 0), dtype=torch.int64)
I think it's weird because the input of llama in Chat3D evaluate function seems to only contain inputs_embeds but I don't know where is the problem, could you please help me to fix it?