The output of `python collect_env.py`
```text
torch 2.4 cuda.12.4
```
Model Input Dumps
for k, v in llm.llm_engine.model_executor.driver_worker.model_runner.model.named_parameters():
v.data.copy_(state_dict[k ])
outputs1 = llm.generate(prompts, sampling_params)
v.data = state_dict[k ] # k is final_layernorm.weight
outputs2 = llm.generate(prompts, sampling_params)
outputs1 is ok and outputs is messy,but the parameter value are the same.
Why v.data.copy_(state_dict[k ]) and v.data = state_dict[k ] would lead to different generation result?
🐛 Describe the bug
for k, v in llm.llm_engine.model_executor.driver_worker.model_runner.model.named_parameters():
v.data.copy_(state_dict[k ])
outputs1 = llm.generate(prompts, sampling_params)
v.data = state_dict[k ] # k is final_layernorm.weight
outputs2 = llm.generate(prompts, sampling_params)
outputs1 is ok and outputs is2 messy,but the parameter value are same.
Why v.data.copy_(state_dict[k ]) and v.data = state_dict[k ] would lead to different generation result?
Before submitting a new issue...
[X] Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.
Your current environment
The output of `python collect_env.py`
```text torch 2.4 cuda.12.4 ```Model Input Dumps
outputs1 is ok and outputs is messy,but the parameter value are the same. Why
v.data.copy_(state_dict[k ])
andv.data = state_dict[k ]
would lead to different generation result?🐛 Describe the bug
outputs1 is ok and outputs is2 messy,but the parameter value are same. Why
v.data.copy_(state_dict[k ])
andv.data = state_dict[k ]
would lead to different generation result?Before submitting a new issue...