DOI-USGS / lake-temperature-lstm-static

Predict lake temperatures at depth using static lake attributes
Other
0 stars 3 forks source link

Add LSTM Model classes #25

Closed AndyMcAliley closed 2 years ago

AndyMcAliley commented 2 years ago

This PR adds classes that define the LSTM and the EA-LSTM. Closes #19.

The first commit copies the model classes directly from the code supporting Kratzert et al., 2019. The second commit makes the necessary modifications to the classes, so the diff for that commit should be useful for reviewing.

Three classes

There are three classes defined: LSTM, EALSTM, and Model. The Model class is a wrapper for both the LSTM and the EALSTM classes. It's a convenience class that allows a user to use either an LSTM or a EA-LSTM based on the boolean inputs concat_static and no_static.

My modifications to the classes

I modified the classes to achieve two goals that the original code wasn't set up to achieve.

  1. Provide predictions at every time step. The original code only produces a prediction for the final time step.
  2. Allow multiple output values at every time step. This is needed because we want to predict temperatures at multiple depths at once.

Additional context

Right now there's no code that instantiates objects from these classes. I'll submit another PR that trains the model, so there will be an opportunity to see the Model class in action then. Some feedback could wait for that PR. I can provide (messy!) code to play with the model classes if desired.

These classes were originally found in the repository https://github.com/kratzert/ealstm_regional_modeling/ and modified. They were written in support of

Kratzert, F., Klotz, D., Shalev, G., Klambauer, G., Hochreiter, S., and Nearing, G.: Towards learning universal, regional, and local hydrological behaviors via machine learning applied to large-sample datasets, Hydrol. Earth Syst. Sci., 23, 5089–5110, https://doi.org/10.5194/hess-23-5089-2019, 2019.

Some modifications are similar to those found in https://github.com/jdwillard19/lake_conus_surface_temp_2021. Those modifications support the publication

Willard, J. D., Read, J. S., Topp, S., Hansen, G. J., & Kumar, V. Daily surface temperatures for 185,549 lakes in the conterminous United States estimated using deep learning (1980–2020). Limnology and Oceanography Letters, https://doi.org/10.1002/lol2.10249, 2022.

jdiaz4302 commented 2 years ago

Left specific comments. Notation comments are more or less take/leave, I don't feel very strongly about them but I gave my thoughts. I followed the matrix shapes and calculations, everything corresponds to the equations.

Curious about the bigger weight initialization comment that I left, but I'm not too concerned about it either - if it works in practice, it's fine; it might be worth looking at if you have trouble training the models at some point though