showlab / MotionDirector

[ECCV 2024 Oral] MotionDirector: Motion Customization of Text-to-Video Diffusion Models.
https://showlab.github.io/MotionDirector/
Apache License 2.0
850 stars 54 forks source link

Why is the GPU utilization low? #6

Open XiaominLi1997 opened 11 months ago

XiaominLi1997 commented 11 months ago

Hi, thanks for releasing the nice work. I running the code, but find that the GPU utilization is very low, is your GPU utilization also low? image Training speed is a little slow. image

ruizhaocv commented 11 months ago

Hi Xiaomin,

I tested the code on A5000, and the GPU usage is nearly 100%. And the training speed is around 1.6s/it. Have you modified parameters other than the file path in the config file?

968d24efcb3184afca2d18d9ec1b0d1 08c106cc6b83149ebafe0eacee500c1

XiaominLi1997 commented 11 months ago

Let me check it, thanks.

ruizhaocv commented 11 months ago

Hi Xiaomin. Have you found the problem?

XiaominLi1997 commented 11 months ago

Hi Xiaomin. Have you found the problem?

hi, I trained on modelscope, and disable gradient_checkpointing, the GPU usage is always unstable. image