search

insight-lane / crash-model

Build a crash prediction modeling application that leverages multiple data sources to generate a set of dynamic predictions we can use to identify potential trouble spots and direct timely safety interventions.
https://insightlane.org
MIT License
113 stars 40 forks source link

Fixes to model train/test and consistency #125

Closed terryf82 closed 5 years ago

terryf82 commented 6 years ago

@bpben I'm a bit unsure of how we start on this one - maybe we should first define what we think a minimum worthwhile probability score is, and then we can address how we deal with the situation where we can't generate that?

bpben commented 6 years ago

Yeah, I think ROC <=.5 is a hard limit, either equal to or worse than chance. Quick fix would be to just stop the prediction process and output error if it doesn't pass that assertion.

bpben commented 5 years ago

A warning has been implemented, moving over to another, higher-level issue: #170