Open andrewfagerheim opened 1 year ago
Below is my work from the whiteboards, with a test-case also included. I think my general thoughts are that it would be better to switch to the EKE system of definitions because it minimizes the magnitude & frequency of negative variance values, where EKE has negative values anytime a filtered profile at a higher scale has a value less than a filtered profile at a lower scale. The next step is to try implementing this method with the applications considered above, namely the ratio method and Steinberg Figure 5 plot.
<img src="https://github.com/ocean-transport/argo-intern/assets/106624836/2aa0b4d6-01d5-4819-ab64-29a6f83b9c0d" height="325"> <img src="https://github.com/ocean-transport/argo-intern/assets/106624836/dc807cdc-712d-49b0-b898-85167a323cc4" height="325">
To better understand the output of each ratio, consider the profiles (i, j, k) of density and spice below
This is the ratio of everything below the smallest scale over everything above the largest scale, ie
$R{ex} = EKE{l=1} / MKE_{l=N}$
As expected, the spice ratio has higher variance, and there's so much variability in the profile that it's very difficult to interpret -- likely because there is so little variance below l1=100m. Density has a lower variance, but it's easier to pick apart
This is the ratio of everything below one scale over everything above that scale, ie
$R{part} = EKE{l} / MKE{l}$
Interestingly, R{part, l=1} looks very similar to R{ex} which is likely because they have the same numerator and the denominator changes from being MKE{l=1} to MKE{l=3}. Also of note, it seems like as filter scale increases for R{part}, the general magnitude and shape of the curves are preserved, but the noise decreases, making it much easier to read the plots.
To better understand the differences in MKE and EKE plots (based off first results in the steinberg notebook), I took a much closer look at each of the terms being calculated by using the substitution u = <u> + u'
. One mistake a made initially that is good to remember is, <<u>u'> DOES NOT EQUAL <<u>><u'>
.
Below is the whiteboard with the substitution into a one-scale EKE/MKE division, and a binning method with multiple scales. Note the color coded underlines denoting terms for large scale, cross terms, and small scale variance. I'll add the plots I've made testing this decomposition when I've finished checking everything over.
I made a new notebook (argo_box_analysis) to walk through all steps in the process of loading a new box and analyzing it with all the methods we've discussed. The goal was to get everything in one place, make sure everything has been done correctly, and organize any questions/errors/etc. This note is to walk through each part of that notebook and note comments/questions here.
I picked a small box because I was hoping this might allow for a more cohesive seasonal signal to appear and the Southern Ocean because I was hoping it would have seasonal variation.
mean_prof = ds.isel(pressure).mean()
and anom_prof = ds.isel(pressure) - mean_prof
bound=True
and apply the boundary mask, which applies the upper and lower bounds as described above. The plots of MKEs and EKEs are displayed, which shows the masking for each.The problem is that float sampling rates can change much more dramatically than I expected or really accounted for in my current method of loading boxes. If the rate changes from ~2m to ~5m that's probably fine, but ~100m definitely isn't. I see a few potential solutions:
Initially, I'm drawn to the first option because it's easy and I know exactly how to do it. However, I'm worried that it could remove lots of profiles, which would not be great for the analysis. However, with the other two that could retain every profile within the box, I don't know how exactly to convert from N_LEVELS (the dimension sampling rate would be on) to PRES_INTERPOLATED (the end dimension for depth/pressure) because N_LEVELS depends on the sampling rate.
Now that I say this though, I feel like if you added sample rate as a coordinate it would be interpolated just like any other coordinate in the ds, so this is trivial anyway. (Wait, but won't it linearly interpolate between each sample rate?? No, I don't think this is going to work...)
I think the way forward then is to load a box where you calculate sample rate, add it as a coordinate, and see if it's interpolated correctly. If it is, check how many profiles would be usable. If not, you can either keep troubleshooting or just remove any profile with a sample rate greater than 6m anywhere.
More on cache & chunks: https://argopy.readthedocs.io/en/latest/performances.html Notebooks: problem_floats and problem_errdac
Notes:
Blackboard notes from https://github.com/ocean-transport/argo-intern/issues/13#issuecomment-1917872869 with @dhruvbalwada
The main Project Organization issue (#10) is getting clogged with notes on specific topics, whiteboard brainstorms, etc so I made this issue dedicated them. #10 will remain dedicated to meeting notes, action items, checklists, etc --> more broad information on the project's direction and management.