Open mengmeexix opened 3 months ago
Do we not need to synchronize before the CPU side modification to ensure that the GPU operator has used up the data?
Which GPU operator are you referring to?
Do we not need to synchronize before the CPU side modification to ensure that the GPU operator has used up the data?
Which GPU operator are you referring to?
Thank you for your reply. I just checked the code and it seems that lmdeploy did not manipulate this data on the GPU. We have added another kernel ourselves, so for our kernel, we can only modify the data after it has been executed.
Checklist
Describe the bug
Here, the value of h'input'length'buf_ is modified on the CPU side, but there was a GPU operator still using this value for calculation. Do we not need to synchronize before the CPU side modification to ensure that the GPU operator has used up the data?
Reproduction
python ./benchmark/profile_throughput_audio.py llama model
Environment
Error traceback
No response