Cassie from Scratch - Githubissues

farzadab / walking-benchmark

How hard is it to walk

0 stars 0 forks source link

Cassie from Scratch #37

Open farzadab opened 5 years ago

farzadab commented 5 years ago

[x] no residual + RSI (+ imitation reward + state) ➜ shuffling
[x] no residual - RSI (+ imitation reward + state) ➜ shuffling
[x] (3D) no residual + RSI (+ imitation reward) ➜ crazy tip-heel?
[x] (3D) Constant residual - RSI (+ progress reward)
[ ] (3D) Constant residual - RSI (+ imitation reward)

farzadab commented 5 years ago

Video: crazy tip-heel with no residual action + RSI + imitation reward (3D).

[env commit ID: 6cf6754] [experiment ID: 2019_06_04__13_35_30__cassie-nomocap]

It seems like the positive reward of speed, orientation, and CoM are too high, so it can get away with not imitating the joint motion ➜ adjusting the weights.

Bug: when residual_control is False, having a DeviationRew does not make sense.

farzadab commented 5 years ago

Video: crouched walk with constant residual - RSI + progress reward (3D)

[env commit ID: be44328] [experiment ID: 2019_06_05__17_22_31__cassie-shinspring]

Does a weird movement with the left leg at the start.

farzadab commented 5 years ago

Video: improved the tip-heel by using more imitation reward, less energy use, no deviation reward, and some initial velocity.

[env commit ID: 8781941e] [experiment ID: 2019_06_07__15_04_43__cassie-rsi-rew6-speed]

farzadab commented 5 years ago

@ZhaomingXie pointed out a bug: if the policy uses a Tanh at the end it can't reach all possible positions.

farzadab commented 5 years ago

Ideas for debugging after talking with @mvandepanne:

Check if mocap is possible with the current joint limits
Hold cassie in the air
- run PD
- learn policy
- use to debug

Other ideas:

torque rather than PD control
curriculum TS?