search

USGS-R / mda.lakes

Wisconsin Lake Modeling Aggregation
2 stars 11 forks source link

2016-03-19 Cal/Bias Results update #87

Open lawinslow opened 8 years ago

lawinslow commented 8 years ago

Ok, this isn't apples to apples with our old model runs. The lakes are different (MI and MN added). I also don't have all the data in here yet.

@jread-usgs you won't be stoked about this though.

Model Bias RMSE
NLDAS 1.83 4.03

Not sure what's going on. ACCESS running right now and digging in a bit into what's going on with NLDAS. I still need to make sure I'm overlapping with all our Wisco lakes, plus the new ones.

jordansread commented 8 years ago

Ugg...with clarity?

lawinslow commented 8 years ago

Yeah, with the clarity data we have. What I don't get is the bias for NLDAS. Needs more investigation.

jordansread commented 8 years ago

Yeah, that is really odd. Makes me worried that the nhd_lat/lon stuff might be shuffled, although maybe that wouldn't create a bias.

Also, I don't think this explains it, but I didn't remove any of the errant data from the temp profiles that were PR'd into lakeattributes yesterday. There were some points that were like 1200°C...which would hike up our RMSE if they weren't removed.

maybe we should check a few north and south WI lakes to compare WBIC_driver.csv to the nhd_driver.RData to make sure this isn't coming from flaws in drivers.

lawinslow commented 8 years ago

Oh yeah, I didn't know there were temps like that. Would definitely skew the RMSE and especially bias.

jordansread commented 8 years ago

But those would pull bias negative, right?

lawinslow commented 8 years ago

Yeah, wasn't really worrying about direction. Just assuming if there are 1200 deg C in there, then we've probably got -1200 too. I'm going with the "big outliers could cause such problems" theory, should investigate. But if not, then sure, bad theory.

jordansread commented 8 years ago

Any way to have that kick out a 1:1 fig w/ the RMSE and bias numbers too? That would help us see if this is a major bias issue, or a bunch of really bad eggs.

BTW, I will PR a change to the way we do the temperature processing, and will remove outliers before it gets to lakeattributes next time.

jordansread commented 8 years ago

Unless you want to have that happen in lakeattributes between data-raw and the package data.