NOAA-PSL / model_error_correction_with_ai

0 stars 2 forks source link

clarify batch_training in README (or comment) #42

Open lslivins opened 8 months ago

lslivins commented 8 months ago

In the README:

3.b) modify batch_training_parallel_GPU.py specifically, in the list below you might want to modify the length of the training period ptmp=[device, 't', 4, '1', '4096', 3, 0.25, 32, 'mse', 0.0001, 1., 366, 365, 0.7]

e.g. 366 above refers to the end of the training period in days and 365 refers to the begging of the trainign period in days. the training period is in reference to the data record stored in the numpy files generated by preprocess.py and spcified by varible "dates"

Can each entry in "ptmp" be explained or commented? I don't understand how "366" is the end of the training period and "365" is the beginning of the training period. What needs to be modified for different training period lengths?

@frolovsa

frolovsa commented 8 months ago

366 is the number of days from the beginning of the dataset where the training period ends. the beginning of the training period is 366-365. E.g. we used day 1:366 for training

another example. if the end is 366, but the length is 6, then we use days 360:366 for training.

the parameters from this list will be passed to the train function here as essentially positional arguments https://github.com/NOAA-PSL/model_error_correction_with_ai/blob/07dd1a7b27c83fc699ed047c52ffdf0d7d45d10f/training.py#L45

we can chat more tomorrow am.

lslivins commented 7 months ago

So instead of "365 refers to the begging of the training period in days", is it more accurate to say that "365 refers to the length of the training period in days" ?