Starting Code for Training

[lordNil]

Hello

I have recently installed and started training with this system. The robot shape and mass was modified in the .xml file to match with the robot in our lab.

For starting the training, in the dp_env_v3.py file, and in the step function:

• the variable "data" was not define, and I assumed it is "self.sim.data"
• should I uncomment the code for self.step_len ? Does this change the simulation step size or something?

I have been trying to train a robot to walk with: mpirun --allow-run-as-root -np 32 python3 trpo.py (my cpu has 32 threads) After 1 million time steps, it usually does something to slowly fall forwards. or stand still. Any tips for training?

Have you successfully trained walking or other advanced moves with this set of code?

[mingfeisun]

Heyo, thanks for your interest. I did train many advanced motions with this codebase. You can refer to this issue to find out one version that succesfully trained walking moves.

[lordNil]

Thank you very much for the help.

I successfully got my robot ( with a heavy backpack, and light legs) to walk. Half of the time it falls, but perhaps survival reward will help.

Time to add some sensor noise and backlash, and see if it still works.

Also, because the network is trained to match the Mocap, would it be possible to train the network first using supervised learning to match the Mocap. Then this network can be trained in simulation to match the robot using TRPO or PPO?

[mingfeisun]

Sure. You can use the supervised learning to pre-train the network and then re-train it with the simulation using TRPO/PPO. It will speed up the training.