GPU usage is low when training

[Haihan-W]

When I run CPO, and SCPO on different tasks, e.g. Goal_Point_8Hazards, Goal_Point_8Ghosts, I realized that the GPU usage is very low, around 1-3%, and the training is very slow. For example, for task of Goal_Point_8Ghosts, it tooks almost one day to run 120-130 episodes.

I verified that my GPU device is available using torch.cuda.is_available(), and checked that the code used GPU as device: device = torch.device("cuda:0" if torch.cuda.is_available() else "cpu")

Any explanation to it is greatly appreciated, or if there is any solution to speed up the training, maybe by increasing the usage of GPU? Thank you!

[LiangZhisama]

Thanks for your question! Based on your feedback and our testing, for complex ghost obstacles, the running speed is indeed as you measured. This is mainly due to the fact that Mujoco3 runs the environment on cpu, while only actor-critic runs on cuda. Environment running speed and resource type conversion occupy the main time.

To address this problem, we are developing a new, completely GPU-based guard framework, GUARDX. It uses the latest MJX GPU acceleration framework released by Mujoco3. But unfortunately, MJX has many deficiencies in friction and other aspects, which prevents most of our existing tasks from running correctly. Therefore, we are constantly improving GURADX while waiting for official updates, hoping to let everyone use a faster framework as soon as possible. The basic tasks have been tested and can be trained in just a few minutes to dozens of minutes.

[Haihan-W]

That's awsome to hear, thank you so much and looking forward to seeing the GUARDX in the future!

Thanks for your question! Based on your feedback and our testing, for complex ghost obstacles, the running speed is indeed as you measured. This is mainly due to the fact that Mujoco3 runs the environment on cpu, while only actor-critic runs on cuda. Environment running speed and resource type conversion occupy the main time.

To address this problem, we are developing a new, completely GPU-based guard framework, GUARDX. It uses the latest MJX GPU acceleration framework released by Mujoco3. But unfortunately, MJX has many deficiencies in friction and other aspects, which prevents most of our existing tasks from running correctly. Therefore, we are constantly improving GURADX while waiting for official updates, hoping to let everyone use a faster framework as soon as possible. The basic tasks have been tested and can be trained in just a few minutes to dozens of minutes.