search

damellis / ESP

The Example-based Sensor Predictions (ESP) system applies machine learning to real-time sensor data.
BSD 3-Clause "New" or "Revised" License
224 stars 52 forks source link

Ability to indicate ground truth data in test data. #144

Open damellis opened 8 years ago

damellis commented 8 years ago

So that we can score pipelines based on how it does relative to that ground truth information. This is probably complicated, as the form of ground truth data may change based on the pipeline / classification type as well as the post-processing done. For instance, we may want to have a way to mark ground truth as regions / intervals and score pipelines against the percentage of the intervals that match. But we might also want a way to specify ground truth as events and score the pipeline by distance between its events and the ground truth ones.

damellis commented 8 years ago

We'll also need a way to save and load ground truth data. I'm not sure if the GRT currently has a conception of this or not.