Closed cmiller8 closed 5 years ago
I updated the cross-validation to take the months in the order that they exist in the individual building data set - ie: if the building data starts in November then the months start with 11, 12, 1,...
Let's make the n
for the cross-validation be 3 and create a metric output file for each of the three models
The final visualization will have 4 sets of metrics - one for the original test/train scheme (every fourth month) and the three cross validation steps (3/3, 6/3, 9/3)
@talantbekov123 - to be clear, can you set n
=3 for the cross-validation and create separate metrics output files for each of those runs?
@cmiller8 this file(https://github.com/buds-lab/building-prediction-benchmarking/blob/master/cross-validation/generate-metrics-cross-validation-steps.ipynb) will create separate metrics output files for each of those runs and store produced output here (https://github.com/buds-lab/building-prediction-benchmarking/tree/master/cross-validation/results-timeseries)
@cmiller8 this file(https://github.com/buds-lab/building-prediction-benchmarking/blob/master/cross-validation/generate-metrics-cross-validation-steps.ipynb) will create separate metrics output files for each of those runs and store produced output here (https://github.com/buds-lab/building-prediction-benchmarking/tree/master/cross-validation/results-timeseries)
Should I create a visualization for this? Like you did in visualizations folder
We can work on the visualization later -- focus on getting the annual schedules as an input first.
Implemented and tested in the master
notebook
Use
sklearn.model_selection.TimeSeriesSplit
to segment the data into training and testing for cross-validationhttps://scikit-learn.org/stable/modules/generated/sklearn.model_selection.TimeSeriesSplit.html#sklearn.model_selection.TimeSeriesSplit
this will be in its own notebooks and will be a fork of the previous scenario with 9 months training and 3 months testings