search

Improbable-AI / walk-these-ways

Sim-to-real RL training and deployment tools for the Unitree Go1 robot.
https://gmargo11.github.io/walk-these-ways/
Other
492 stars 129 forks source link

Actuator Network Training #2

Closed Kashu7100 closed 1 year ago

Kashu7100 commented 1 year ago

Thank you again for this repo.

I am wondering if you could share the data collection/training code for obtaining the actuator network. Since the original paper uses the Anymal robot, which has actuators at its joint, the joint torque and the actuator torque should be the same. However, in Go1, the knee joint is driven by an actuator connected by a rod, thus the joint torque and the actuator torque are not equal in the knee, and I think they need some extra care. I would appreciate it if you can share with me how you trained your actuator network for Go1.

gmargo11 commented 1 year ago

Hi @Kashu7100 ,

I plan on adding the data collection code, data, and script for actuator network training to this repository. It might happen in a few days. I will leave this issue open until that code is available.

The actuator network is trained to predict the output torque at the motor (estimated by the Unitree SDK) from a short history of the joint position errors and velocities. This doesn't account for the dynamics of the knee linkage. Ideally, we'd train a separate model to predict the output torque at the knee. Unfortunately, since there's no torque sensor in the knee, the data isn't actually available to do this. So, we shared the same actuator network among all 12 actuators. There's definitely room to improve here as you suggest.

One suggestion is to incorporate the knee joint dynamics in post. The MIT mini cheetah used a belt transmission which resulted in an effective torque reduction and different limit on the torque bandwidth that was characterized in https://dspace.mit.edu/handle/1721.1/118671. In principle, the Unitree knee joint should have a similar effect due to its bar linkage. The doc here https://www.wevolver.com/specs/unitree-robotics-go1-edu-plus indicates that the gear reduction for the knee increases output torque limit on the real robot by 1.5x. So my first inclination would be to try multiplying the actuator network output by 1.5x during training. Accounting for the bandwidth reduction would be interesting too and could maybe be done using data from the motor.

Kashu7100 commented 1 year ago

Thank you for your clear response. I am looking forward to seeing your code.

As for the knee joint, besides the gear reduction, there will be a reduction in linkage so the final reduction ratio might be different from 1:1.5. I will double-check with Unitree and get back to you.

PS: for the lcm_position, could you please check the header files (detailed in the previous issue)?

gmargo11 commented 1 year ago

Just added a working fork of the unitree_legged_sdk that builds lcm_position successfully on my Go1: https://github.com/Improbable-AI/unitree_legged_sdk

Yes, if you find out more details from Unitree they'd be welcomed :)

gmargo11 commented 1 year ago

Update: Commit 5d5cb24f97cf4642c020d1338ff231737cbb5866 adds actuator network training and evaluation scripts to the scripts/actuator_net folder. Hope this makes the architecture transparent and enables users to experiment with different approaches.

[Concerning the training data for the actuator network]: I don't think the code in its current state generates new logfiles on the robot during operation, so for now I provided an example logfile from my robot that can be used to train actuator nets. Adding good documentation for generating logfiles is high priority for this repo, but I likely won't get around to it for about two weeks due to travel.

sharedcare commented 1 year ago

Hi @gmargo11,

I want to train an actuator network for Unitree A1. How can I collect data for A1 to train the actuator network? I think there is not much difference between A1 and GO1.