Cross-validation improvements

ccao-data / model-res-avm

Automated valuation model for all class 200 residential properties in Cook County (except vacant land and condos)

GNU Affero General Public License v3.0

26 stars 5 forks source link

Closed dfsnow closed 8 months ago

dfsnow commented 8 months ago

After some discussion, we should make the following changes to cross-validation:

[x] Revert to rolling-origin splitting (we temporarily went to v-fold)
[x] Make the rolling-origin non-cumulative
[x] Store the predictions and residuals by validation set, so we can get a sense of how error changes over time
[ ] Try slightly reducing the time span of the training data (currently 9 years, try 6-7)