predicting VIC soil moisture based on NLDAS Forcing A

[PeterDSteinberg]

• Goal of example: Recreate the VIC soil moisture field using the NLDAS Forcing A (FORA) data set. Train on one hour’s FORA and use the trained model to predict the next hour’s VIC soil moisture.
• Use Elm / Earthio Pipeline to transformers:
  • Time integration and differencing for various intervals over preceding 12 days
  • Scaling
  • Log Transform
  • PCA - Choose the best number of components
  • Select regression models based on: MSE, R2, bounds checks for soil moisture
  • Build a second layer regression model on the selected models’ predictions
  • Holoviews viz

cc @jbednar @gbrener @gpfreitas

[PeterDSteinberg]

TODO reminder for myself:

nldas_soil_moisture_ml.py:432:5: F841 local variable 'X_time_steps' is assigned to but never used
nldas_soil_moisture_ml.py:444:5: F841 local variable 'last_hr' is assigned to but never used
nldas_soil_moisture_ml.py:470:5: F841 local variable 'Xnew' is assigned to but never used

[PeterDSteinberg]

This screenshot shows the notebook in this PR:

Currently the script's ML approach to predicting the next hour's soil moisture has a 10% bias relative to VIC soil moisture. The two approaches have predictions with similar central tendencies, but Elm has more pos/neg outliers than VIC. I think that is a common problem in statistical vs deterministic models. I'll use today's demo to come up with improvement ideas.

[gbrener]

@PeterDSteinberg, the color scales now match up. I also equalized the histogram bin ranges to make comparisons a bit easier:

[PeterDSteinberg]

Thanks ! That's a lot more useful now (the fix of the color scales), @gbrener . We should make sure the viz libraries' docs show this kind of thing directly in an easy to find way. It wasn't immediately obvious to me when googling around.

[gbrener]

Yeah, I actually needed @philippjfr's help to make it work (he designed much of the API). He acknowledged that the docs leave a bit to be desired in this area.