search

damellis / ESP

The Example-based Sensor Predictions (ESP) system applies machine learning to real-time sensor data.
BSD 3-Clause "New" or "Revised" License
224 stars 52 forks source link

Need to save timestamps with test data and replay them when analyzing it. #66

Open damellis opened 8 years ago

damellis commented 8 years ago

After re-training the model and re-running prediction on the test data, no correct predictions are made. This happens even if the training data is unchanged (and the model was simply retrained on the same training data). Predictions work on future live data, just not on the test data.

(Trying with the user_accelerometer_gestures.h example.)

damellis commented 8 years ago

This seems to have been the result of a ClassLabelTimeoutFilter post-processing module, which filters out multiple successive predictions. We may need a way to run prediction with fake timestamps. (We could ignore some or all post-processing modules but that will give different results than running the tests live.)

damellis commented 8 years ago

This seems like something we should punt until later, when we look at temporal calibration more generally.