search

USGS-R / river-dl

Deep learning model for predicting environmental variables on river systems
Creative Commons Zero v1.0 Universal
21 stars 15 forks source link

Error when reduce_*_trn parameter is 1 #61

Closed jsadler2 closed 4 years ago

jsadler2 commented 4 years ago

I'm getting an error when I set the reduce_flow_trn or reduce_temp_trn parameter to 1 in the prep_data function. This is because I have an assertion to check if the standard deviation and mean of the training data are real numbers. When I set the reduce_*_ parameter to 1, it replaces all of the variables values with nan, so the standard deviation and mean are not real numbers.

jsadler2 commented 4 years ago

I feel like my code is a little messy in this area. There are multiple ways now to discount the observations:

  1. I can set the pretrain_vars or finetune_vars to either flow or temp (or both). - This sets a per-observation weight to zero for the variable that is not included in either of those parameters. These weights are multiplied by the per-observation RMSEs in the loss function
  2. I can discount a number of segments - Like above, this sets a per-observation weight of zero for the segments I specify
  3. I can reduce the number of observations available. - For this, I am selecting a random number of observations and setting those equal to np.nan.
  4. I can change the temp vs. flow weighting in the loss function.
jsadler2 commented 4 years ago

So I think the easiest actually is just to switch how I do 3. I can just randomly set weights that correspond to observations to zero and that should have the same effect as setting a random number of observations to nan.