Improbable-AI / walk-these-ways

Sim-to-real RL training and deployment tools for the Unitree Go1 robot.
https://gmargo11.github.io/walk-these-ways/
Other
492 stars 129 forks source link

Slower than 2Hz stepping frequency #9

Closed thvhauwe closed 1 year ago

thvhauwe commented 1 year ago

Hey,

Thanks for sharing this code, we find it usefull for actually deployment on the Go1. However, we are struggling to lower the stepping frequency down to lower than 2Hz. Could you elaborate where this limit is programmed?

Things we've tried so far:

gmargo11 commented 1 year ago

Hi @thvhauwe ,

The pretrained policy is only trained with stepping frequency range 2.0 to 4.0, which is specified by the parameters limit_gait_frequency and gait_frequency_cmd_range in https://github.com/Improbable-AI/walk-these-ways/blob/master/scripts/train.py#L157. Training a new policy with both these adjusted, for example to [1.5, 4.0], should work.

You can see how the gait frequency command is converted into a contact target here: https://github.com/Improbable-AI/walk-these-ways/blob/master/go1_gym/envs/base/legged_robot.py#L826

To clarify, is it the case that lower frequency commands are never being sampled when you train, despite changing the Cfg? Or that lower frequencies are being sampled, but the robot is not learning to execute them?

-Gabe

thvhauwe commented 1 year ago

Thnx for replying!

Playing around with the play.py script showed the new policy learned to walk with lower frequency (albeit with some jigging behaviour of swing legs, likely due to untuned rewards).

The remaining question is how to port this to the real robot. What line has to be changed to change the cmd frequency there?

gmargo11 commented 1 year ago

During deployment, the command frequency is computed in go1_gym_deploy/utils/cheetah_state_estimator.py:

Modifying Line 174: cmd_freq = 3.0 will change the frequency in the default mode

Modifying Lines 183-186 will adjust the range of frequencies you can command with the joystick:

if MODE_RIGHT == "step_frequency":
            min_freq = 2.0
            max_freq = 4.0
            cmd_freq = (1 + self.right_stick[1]) / 2 * (max_freq - min_freq) + min_freq

Be sure to use the top right trigger to toggle into mode E (step frequency).