search

CANFLUX / Calculation_Procedures

Ini files for cleaning under ../Database/Calculation_Procedures
0 stars 1 forks source link

Ziyi bc tidal #26

Closed tzuyilu closed 1 year ago

tzuyilu commented 1 year ago
tzuyilu commented 1 year ago

Hi Zoran, For TS1*_1 variables, it will be a good idea to create a flag for them, and I will create one. Just because this flag will be used once, so I put it as an individual line in 2nd stage ini-file. In addition, I would like to discuss the usage of 'run_std_dev' with you. I use this function to remove the spikes that can't be identified in the first stage. Those spikes might fall within reasonable ranges but look like sudden jumps in the series. That's why I think they should not stay. So I think we should use this function (but I should put it before the 'calc_avg_trace' actually) in 2nd stage. What do you think?

June-Skeeter commented 1 year ago

@tzuyilu yes, the run_std_dev should go before calc_avg_trace if it will be used. But we'll want to be mindful of which variables it's being applied too. These are things we should probably discuss collectively so we use the same implementation for all ini files/sites

tzuyilu commented 1 year ago

@June-Skeeter That makes sense to keep consistent across ini-files/sites. We can discuss which variables should be applied at the upcoming meeting. Thanks!

tzuyilu commented 1 year ago

Hi @znesic, I just edited ini files according to our previous discussions. Here is the updated summary of my edits:

Modified shiftMyData.m under 'DSM/Derived_Variables': (1) deleted the line of the 'run_std_dev' (2) added memo about timestamp shift issue

DSM_FirstStage.ini: (1) added 3 new flags ('flag_B4INSTAL', 'flags_PRI_1_1_1', 'flags_NDVI_1_1_1') (2) added RBM_SW_OUT_1_1_1/RBM_LW_OUT_1_1_1/RBM_NETRAD_1_1_1 (for DSM LWout correction)

DSM_SecondStage.ini: Deleted redundant contents (1) deleted line of the 'del_sp_seg' in water measurements (2) fixed Biomet lag issue by 'shiftMyData.m' (3) adjusted the order of run_std_dev/calc_avg_trace functions (4) removed 'rmNoise.m' mentioned in pull request Ziyi bc tidal #23

RBM_FirstStage.ini: (1) corrected a variable name (MET_HMP_RH_4m_Avg) (2) added a new flag ('flag_B4INSTAL', 'flag_TS_1') (3) changed minMax of spikes_hf_2/5/6/7/8 from [0,0.1] to [0,100]

RBM_SecondStage.ini: (1) Adjusted the order of run_std_dev/calc_avg_trace functions (2) removed 'rmNoise.m' mentioned in pull request Ziyi bc tidal #23