Auxiliary Loss weight tuning

[farzadab]

Tune the w parameter in the auxiliary loss method. Environments:

• [x] Walker2D: 2
• [x] Walker3D: 4
• [ ] Humanoid?
• [x] Cassie2D: 4

[farzadab]

Results for Walker2D

• It doesn't seem that sensitive to the weight value
• The best values seems to be 2 though it might be more due to random seed

Experiment IDs:

• 2019_07_10__19_13_36__w2_karen_1
• 2019_07_10__19_13_36__w2_karen_2
• 2019_07_10__19_13_36__w2_karen_4
• 2019_07_10__19_13_36__w2_karen_8

[farzadab]

Results for Cassie2D

• All values seem to perform similarly
• Using w=4

Experiment IDs:

• 2019_07_12__11_42_30__cassie_karen_1
• 2019_07_12__11_42_48__cassie_karen_2
• 2019_07_12__11_42_53__cassie_karen_4
• 2019_07_12__11_42_58__cassie_karen_8

[belinghy]

Results for Walker3D

Fairly similar for all coefficients, 4 is best by very slight margin.

Experiment IDs:

• 2019_07_1220_35_43w3_karen_1
• 2019_07_1220_35_43w3_karen_2
• 2019_07_1220_35_43w3_karen_4
• 2019_07_1220_35_45w3_karen_8

[farzadab]

Conclusion: Overall it seems like the learning curve is pretty insensitive to the weight parameter. We will be using w=4 for most experiments.

Question: How much is the gait/policy symmetry affected by the weight parameter?