IMU Representation Overfitting

[elvout]

Problem

The IMU representation learner is clearly overfitting to the training data. Test data collected in the same manner as the training data does not separate cleanly in the (PCA projected) embedding space.

Training on statistical features only seems to generalize better than training on the full data contained in each time window. It also "spherizes" better, does this signify anything?

Investigate

• [ ] Recurrent NN
  • [ ] LSTM
• [ ] Dropout

[elvout]

Training on only statistics of the "foot_depth_mean" data type:

• Clear separation between grass and {concrete, small rocks} in the test set
• Essentially no separation between concrete and small rocks in the test set

Overall the results were as expected. It's unclear though why this level of separation is lost when training on other data as well.

[elvout]

Training on statistics of the "foot_depth_mean" and "foot_depth_std" data types:

In the test set:

• Begin to see separation between small rocks and concrete, still significant overlap
• Small rocks does not cluster as compactly as concrete

[elvout]

Training on statistics of {foot_depth, foot_slip} data types (dim 3):

In the test set:

• Acceptable clustering and separability of terrain types

Unfortunately, this is training entirely on data collected by Boston Dynamics' stack, although theoretically it could be reconstructed from joint data.