search

conradbm / streamml

Streamlined Machine Learning
MIT License
0 stars 0 forks source link

Model Performance #3

Open conradbm opened 6 years ago

conradbm commented 6 years ago

Model Performance probably needs to be its own class or at least its own method within ModelSelectionFlow. Once your top 5-10 models have been chosen, then put them through a contest on the test data set to see who performs the best on the most # of metrics (using an odd number of metrics), then return the winner/results with some simple statistics discussing how much better % wise that model was.

conradbm commented 6 years ago

Consider a AbstractStatistics class with a ModelStatistics and FeatureStatistics or TransformationStatistics comparing if we did better with any of these options. This would be the place for confidence intervals and pvalues