giantotter / giantotter_public

2 stars 0 forks source link

Make classifier more robust to test-time errors #773

Open cmward opened 7 years ago

cmward commented 7 years ago

Since the cluster classifier uses sequence features, an incorrect prediction at timestep t-1 can lead to an incorrect prediction at timestep t. The current data augmentation method cannot alleviate this, since it will never generate explicitly incorrect feature configurations.

The DAgger method can create a dataset that allows the classifier to learn how to recover from past mistakes by casting sequential supervised learning as an imitation learning problem.

cmward commented 7 years ago

Putting this on hold until after determining whether we should switch to XGBoost.

cmward commented 7 years ago

Pretty sure this can be done with XGBoost + crowdsourced cluster annotations of human-AI logs. Take annotated cluster labels are gold standard, and leave the rest of the features from the log unchanged.