FALL 2022 & SPRING 2023: Roadmap/Project Organization

[andrewfagerheim]

Repo Organization:

• 8 [Collection of meeting notes, action items, and timelines on the direction and progress of the project starting from FALL 2022. See #1 for a similar collection from SUMMER 2022.]

• 3 [Accumulated list of useful articles, websites, code tips, etc starting from SUMMER 2022.]

• 4 [Accumulated list of key points, takeaways, and questions from articles and reading materials starting from SUMMER 2022.]

• Any issue whose title begins with ISSUE describes an actual roadblock or question I have, compared to the issues listed above which are for organizational/admin purposes.

[andrewfagerheim]

7 October 2022: First Steps

Dhruv's notes from Slack:

• glider data has slope change at 100m but Argo analysis doesn't -- should apply both filtering and spectra methods to each dataset and compare
• end goal for filtering and spectra practice is to either 1) get identical interpretations when using each method for the same region OR 2) understand the features/limitations of each method
• load more floats from the Arctic Circumpolar Current (ACC) region as this should have layering
• create function that allows easy transition to depth and density coordinates

Action Items:

• [x] Reorganize repo for this semester (clean up issues, edit README, etc)
• [x] Read AGU funding procedures from Dr. Abbot, look into hostels & flights
• [x] Read Smith & Ferrari 2009 and take detailed notes
• [x] Work through Dhruv's vertical spectra notebook keeping in mind the things mentioned above

[andrewfagerheim]

14 October 2022: Meeting w @dhruvbalwada

Discussion of Smith & Ferrari 2009:

• density/spice coordinates: neither density or spice physically exist, but rather both are created to explain how the ocean is structured; density is an active tracer (impacts flow) that represents how temperature and salinity are structured due to gravity; spice is a passive tracer (doesn't impact flow) that represents temperature and salinity NOT disturbed by gravity
• inverse cascade: tendency of eddies to merge together to form larger eddies; most density variability occurs at large-scales so it is an inverse cascade (see Figure 7 for density spectra); for stirring that only occurs at large scales we see small-scale variability in spice; for stirring at all scales we see interference that doesn't allow small-scale structure to form as easily
• vertical/horizontal scalings: N/f (vertical stratification/Coriolis force, ie horizontal force), this incorporates the timescales of vertical characteristic frequency and planetary rotation; in other words, it's a ratio of the timescales of vertical and horizontal disturbance (see Figure 13 for a comparison of the vertical/horizontal ratio for the flow (sheer/strain), tracer, and physical variables (N/f)
• quasigeostrophic model: assumes that since the horizontal scale of the ocean is so much greater than the vertical scale, only horizontal velocity can be taken into account; shear represents how quickly the horizontal velocity effects vertical structure, strain represents how quickly flows are pulled together/pushed apart horizontally (see Figure 5 for plots of shear and strain across depth)
• Figures 11 & 12: if you can define the right amount of "stretch" to match the scale of the vertical to the scale of the horizontal, the spectra will also line up fairly well, this means if we can measure information in the vertical well, we can relate it to what's happening in the horizontal (see Figure 11 to see how K (wavenumber of horizontal) and λ (adjusted wavenumber of vertical) and Figure 12 for similar plots for EKE)
• theory for future work: if spectra drop off at tens of meters this indicates the influence of internal waves breaking; dropping off at 100 or hundreds of meters indicates horizontal mixing because internal waves won't affect variance at that scale --> this is why Dhruv's glider data dropping off at 100m was significant

Project Connections:

• we'll keep referring back to this paper so keep it handy
• we need to look for a similar drop off in specra for argo, but also figure out why we don't see a drop off in filter method; is this a problem with the filtering method? or a difference between glider and argo data?
• admin item: roughly keep track of hours spent on this project, goal is to work through the grant funding first and then (hopefully) get funding from the group

[andrewfagerheim]

4 November 2022: Meeting w/ @dhruvbalwada

Discussion of vertical_spectra notebook:

• two data files:
  • isopycnal_grid is binned by density, going from 27.0 to 27.9 in 0.01 increments
  • vert_4m is binned by pressure/depth, in 4m increments (glider samples ~1-2m so wanted multiple points per bin, but break is ~100m so resolution is high enough)
• "potential density" is density only dependent on temperature and salinity, the effect of pressure has been removed by assuming the water parcel moved to the surface adiabatically; "anomaly" is this density minus 1,000 or the density of fresh water (because density in the ocean does not very greatly)
• notebook has four methods for assembling the data before conducting spectral analysis
  • [-1] T (z, dive): temperature as a function of depth and dive
  • [1] T (z~, dive): temperature as a function of depth (corrected for horizontal distance) and dive
  • [2] 𝛿 (z~, dive): spice as a function of depth (corrected for horizontal distance) and dive
  • [3] 𝛿 (z~', dive): spice as a function of depth (corrected for horizontal distance and with the variation of internal waves removed) and dive
  • [4] 𝛿 (z~'n, dive): spice as a function of depth (corrected for horizontal distance, the variation of internal waves removed, and the variation of spice along isopycnals normalized)
• all of these methods still show a change in spectral slope ~100m which indicates a very robust signal. The question is why the argo data didn't display similar trends with a filtering approach. Importantly, we must understand:
  • Is this due to limitations of filtering or a special effect of the spectral plots?
  • Is this a unique feature of the glider region that wasn't present in the selected Argo boxes?

Next steps:

• [x] analyze the glider data using a filtering approach and analyze results - do we see a change in slope around 100m? Start with method [-1], using temperature as a function of depth and dive to produce the plots

[andrewfagerheim]

11 November 2022: Meeting w @dhruvbalwada

Notes on argo_vertical_spectra notebook:

• seems like the filtering results look similar to spectral plot (?) next steps will provide more direct comparison
• goal is to pass data from both glider and Argo in the same region through both filtering and spectral approaches and compare --> do both datasets contain the same features? do both methods agree on the structure?

Next steps:

• [x] GLIDER DATA: correct sigma to be in length scale, not grid points
• [x] GLIDER DATA: plot filtering curve on the same axes as spectral curve (need some kind of normalization?)
• [x] ARGO DATA: define an Argo box in the same place as the glider
• [x] ARGO DATA: apply filtering and spectral methods to the Argo data and compare

Long Term:

• compare two methods from two regions and constrain any differences
• move from looking at T (z, dive) (approach [-1]) to 𝛿 (z, dive) (approach [1])

[andrewfagerheim]

18 November 2022: Meeting w @dhruvbalwada

Notes on argo_vertical_spectra notebook:

• seems like the argo data has a weaker signal than the glider, using both methods
  • for spectral, "break" in slopes occurs at lower length scale
  • for filtering, there doesn't seem to be much of a break at all
• maybe this is because argo data has a weaker signal -- averaging over events and times weakens the signal compared to the glider which was only over a few weeks
• solution to break glider down into separate events visually, then see if we can do the same for argo data

Next steps:

• [x] GLIDER DATA: pick 3-5 parts of the path where the large-scale signals look distinct and compare their separate spectra
• [ ] ARGO DATA : write a function to read in "event" parameters and return separate xarrays for each event

Long Term:

• need to reconcile spectra and filtering methods
• eventually move to 𝛿 (z, dive) (approach [1]) space

[andrewfagerheim]

21 November 2022: Meeting w @dhruvbalwada

Notes on argo_vertical_spectra notebook:

• possible that windowing is also playing into this -- if large anomaly occurs at the edges of a window will it be suppressed? how do you turn windowing off (window=False doesn't work)?
• probably time to be more quantitative in our analysis - write function to fit line(s) to each spectral curve and report slope, 'break' point, relate to surface temp or dT
• seems like surface temp is a better predictor of large-scale variance than dT

Next steps:

• [x] GLIDER DATA: examine how windowing effects spectra (window=True is depreciated and window=False gives an error) [@dhruvbalwada]
• [x] GLIDER DATA: more quantitative relationship between spectra and filtering methods [@dhruvbalwada]
• [ ] GLIDER DATA: plot spectral curve for every profile (assign greyscale gradient based on surface temperature) and add average lines of high and low EKE regions [@andrewfagerheim]
• [ ] ARGO DATA: sort profiles by 4 month time blocks and plot spectra/filtering curves -- are there enough profiles to see a clearer signal? [@andrewfagerheim]
• [x] ARGO DATA: sort profiles by average surface temperature and plot spectra/filtering curves -- do we see similar patters to the glider separated by selections? [@andrewfagerheim]

[andrewfagerheim]

27 December 2022: Meeting w @dhruvbalwada

Ideas for next steps:

• the figures on the AGU poster are looking at variance of anomaly, Sadek & Aluie paper has more formalized/standard definitions
• Dhiuv will continue to take a look at filtering/spectral comparison and how to address boundaries
• plot hacky variance of anomaly method in idealized notebook to see if it looks similar at all
• pick just one filter scale (100m) and compare large and small scale variance: which of the five boxes has the highest small-scale variance?
• take idealized notebook's signal, add e^(-x) to make it a decaying function, try above method with splitting at one scale to see how this compares

[andrewfagerheim]

17 February 2023: Meeting w @dhruvbalwada

Immediate Tasks:

• [x] Compare Dhruv's rigorous filtering method from the Filtering_and_Fourier_transforms repo to my functions that use scipy.gaussian_filter1d. Are the results identical?
• [x] If they are not identical (and honestly maybe even if they are), make new ones with the same method as Dhruv's repo
• [x] Use these functions to perform the calculations about "MKE" and "EKE" (from Steinberg et al 2022) on one scale:
  • [x] Pass temperature signal through filter at l=100m, then square the results <-- "MKE"
  • [x] Square the temperature signal, pass it through filter at l=100m, then subtract MKE <-- "EKE"

_Updates can be found in the argo_onescale notebook in my repo

[andrewfagerheim]

24 February 2023: Meeting w @dhruvbalwada

Today I started to update my filtering functions to match the functions that Dhruv wrote. I ended up creating two functions: one_scale_filt for filtering a dataset at just one scale and multi_scale_filt for creating an array at multiple filter scales.

Next steps:

• [x] Match results of Dhruv's functions for Argo data in region of the glider
• [x] Compare with results of my old functions, are they the same [spoiler alert: no]
• [x] Create new function that provides same results as Dhruv's functions:
  • [x] Select filter scales based on length in meters
  • [x] Add option to remove data at boundaries with the same length as filter scale --> create mask variable
  • [x] Add option to select between arranging filter scales linearly and logarithmically

_I've added lots of notes walking through these steps in more detail in the argo_onescale notebook

[andrewfagerheim]

3 March 2023: Meeting w @dhruvbalwada

Notes on argo_one_scale:

• All 3 functions are giving the correct outputs! All flags are working correctly (lin/logspace, boundary_correction)!
• Not written the most elegently, ie should be split up into simpler tasks (like one function to create mask, one to get filter scales, one to perform filtering, etc.). Each function should ideally perform one task instead of an entire process.
• Also functions should work for any type of data we give them in a similar format, ie they should work for both glider and argo data. This means that any argo-specific variables like TEMP or PRES_INTERPOLATED should be written as flags instead.
• Note for the future: currently filter.gaussian_filter1d is working okay because it automatically assumes the axis is -1, in the future this could be an issue of the array entered doesn't have the dimensions ordered correctly. To solve this, you would use xr.apply_ufunct to apply the filter to the correct dimension.

Next steps:

• [x] Differentiate between variable (temperature, salinity, spice, etc) and dimension (pressure)
• [x] Split different tasks into separate functions:
  • [x] Create mask (toggle between boundary_correction==True and False)
  • [x] Create filter scales (toggle between arranging on logspace and linspace)
  • [x] Filter data (function to filter at multiple scales should call function for single scale)

[andrewfagerheim]

10 March 2023: Meeting w @dhruvbalwada

Notes on argo_one_scale:

• Functions look good!
• Should add '''Docstring''' to describe functionality, what should be given to the function, what it will return, what different keywords mean, etc
• EKE seems to convey expected trends for this region, namely high small scale variance in the mixed layer that decrease quickly after ~750m. Should try this method in different regions to see what they look like.
• Potential problem with filtering method: functions currently use 'wrap' mode in gaussian_filter1d which is likely what's causing the "tail" at depth. At some point we'll have to manually write a new function to instead use the mirroring method. (This will probably double the length of the domain to [-2000m, 2000m] and filter on that signal.)

Next steps:

• [x] Add docstring to functions with good descriptions
• [x] Compute EKE on other regions and compare.
  • [x] Try four regions from AGU poster
  • [x] Try Southern Ocean regions from Sallee paper
  • [x] Create new functions for computing EKE (see EKE_funcs.py)
  • [x] Compare argo results to glider data? (tried, but the glider dimensions are opposite argo's; need to use apply_ufunc?)
  • [ ] Try something along the equator?
• [ ] Take a stab at mirroring function instead of gaussian_filter1d? [probably more long term]

[andrewfagerheim]

14 March 2023: My Notes

Notes on progress:

• EKE functions look good! Plotted results of EKE for temperature and salinity next to the average profiles for each in two sets of regions (the four boxes from the AGU poster and 6 evenly spaced around the Southern Ocean)
• EKE is a precisely defined quantity, I'm not sure what to call the quantity I've been calculating
• Something doesn't look right with the salinity EKE plots in the argo_one_scale notebook, why are there lots of horizontal lines?
• All of the functions I've written so far will not work for the glider data because the dimensions are backwards! See the March 3 notes on the expected dimensions of gaussian_filter1d filtering on the -1 axis, which in argo is 'PRES_INTERPOLATED' and in the glider is 'dives'

Potential next steps?

• [x] Write docstring for new EKE functions
• [x] Check what's wrong with EKE salinity plots (check salinity profiles, use larger filter scale)
• [x] Pick larger box in glider region, split into seasons, and perform analysis. Do seasonal data differ?
• [ ] Check if functions work with glider data (transpose dimensions before using function)
• [ ] Eventually try to write mirroring method instead of wrap for filtering

[andrewfagerheim]

17 March 2023: Meeting w @dhruvbalwada

Notes:

• Salinity EKE plots could look strange because something with the profiles themselves are off, or because they are too smooth that there is virtually no small-scale variance (so you're subtracting very small values and getting something weird from it).
• For the glider data issue, should transpose dimensions first and then give the function an array with the pressure dimension last. Use xr.transpose() to swap dimensions.
• Also should look at expanding the analysis across time (in addition to spatial bounds with looking at different areas). Pick a much larger region around the glider (maybe 20lon x 10lat?) and split the year into four sections by month using ds.TIME.dt.month. Plot the mean temp/salinity and EKE signals for each month. Do they look different?

Next steps:

• Updates added to the list above, since I haven't made much progress on that yet.

[andrewfagerheim]

24 March 2023: Meeting w @dhruvbalwada

Standardized terms: <A>l is the profile filtered at scale l <A>l^2 is the profile filtered at scale l, squared [MEAN VARANCE, MV]

A^2 is each value of the profile squared <A^2>l is each value of the profile squared, filtered at scale l [TOTAL VARIANCE, TV]

<A^2>l - <A>l^2 is the difference between TV and MV [EDDY VARIANCE, EV]

Notes:

• Potential (and interrelated) factors contributing to salinity plotting issue: fewer profiles, homogenous profiles, smaller filter scale, near vertical curve after a certain depth
• In other words, the more uniform salinity profiles in a region are, the more eddy variance will tend toward zero (and therefore not plot well)
• Would be interesting to check other regions in the Southern Ocean, maybe pick 2-3 other large boxes and create the same four plots that are in the argo_seasonality notebook. Do they show similar trends? Also plot the years and profile numbers of profiles in this region. Are we getting multiple times and floats represented, or is most of our data from one sampling event?
• Next step is to move to density space, review the methods from the Balwada paper using the notebook clone. Particularly look at the z_grid_to_isopycnal_grid and Figure2-sections_plot notebooks!

Next Steps:

• [x] Plot year and float number for 20x10 Southern Ocean box
• [ ] Compute density on the 20x10 SO box. Create an xarray with dimensions converted to rho_grid
• [ ] Calculate eddy variance of density. How does it compare to temperature and salinity?

[andrewfagerheim]

31 March 2023: Meeting w @dhruvbalwada

Notes:

• Should use argo.teos10() to calculate CT, SA, SIG0; this will be easier than using gsw directly. Should try adding these variables to the dataset before using argo.point2profile() or interpolating in the argo_box_loader notebook
• Before the Climate School poster event, goal should be to have plots of
  • mean temp/salinity profiles in depth space
  • mean eddy variance in depth space
  • mean eddy variance in density space
  • probably other plots that help visualize the depth --> density conversion
• QUESTION: I've been selecting profiles with average sampling rate > 4m, but interpolating on a 2m grid. This probably isn't smart right? About how many actual points should be in each pressure bin?

Next Steps:

• [x] Modify argo_box_loader notebook to include CT, SA, SIG0 variables in output xarray
• [x] Explore the z_grid_to_isopycnal_grid notebook, start to replicate for argo data
• [x] Read Dove paper. Pay special attention to Figure 2

[andrewfagerheim]

7 April 2023: Meeting w @dhruvbalwada

Notes:

• Seasonality plots: Differences are close to the surface, at depth density and EV are very similar. This is likely because narrowing our scope to one geographic region means we're looking at roughly the same hydrological conditions. The only thing changing then is seasonal impacts of temperature, snowmelt, etc
• Spatial plots: For the boxes spaced around the Southern Ocean, we're cutting in and out of the ACC which is what leads to the variability between profiles. The ACC is located farthest north (well north of 60S) east of South America and located farthest south (south of 60S) west of South America. Boxes north of the ACC have gentle density changes, those in the ACC have more definition, and the one south of the ACC has the steepest change in isopycnals.
• INTERPOLATION: Probably good to have finer resolution in interpolation levels than actual data, this will ensure we're not cutting out any points (which could lead to a smoother signal than what was actually sampled). So definitely 2m is better than 4m, might even want to try 1m instead.
• For EV density plots, we should divide the EV of density by the vertical gradient of density (d_rho/dz), however when we tried there was still a significant amount of noise in the system. Instead, we should use a lower resolution average by binning with the following technique that needs to be made into its own function
  • density is currently organized on a 2m grid
  • use .coarsen().mean() to move to a rougher grid (20m? 30m?)
  • use .diff() to compute d_rho
  • use.interp() to return to 2m spacing of density profiles and calculate d_rho/dz
• Look at Roullet paper's plots as ballpark idea of what our zeta values should look like

CCS Poster:

• Motivation: Similar to previous posters, about lateral stirring and its impacts on vertical structure
• Dataset: Basics of Argo data, not sure if the float cycle figure is necessary (I like it because it shows how floats collect samples which is relevant to its vertical/lateral sampling resolution, but that's also a lot of real estate.)
• Methods: Similar to previous posters, except show how TV, MV, and EV are calculated for one profile instead of anomaly/variance
• Results: Should be split to show both spatial results (4 regions and EKE or 6 regions and ACC?) and temporal results (big SO box split by month). Should have 4 panels: mean temperature, mean salinity, mean density, and mean EV of density (using the updated function).
• This is the first version, a complete draft should be ready for Dhruv to look over by April 14 and should be completely finished and submitted by April 19. If I make decent progress on the density analysis before April 28, then maybe I can print another version at Lamont and bring it.

Next steps:

• [x] Have draft of CCS poster registration ready for Dhruv to look through
• [x] Have draft of CCS poster itself ready to review with Dhruv
  • [x] Create a function for computing d_rho/dz and use it in the EV of density plot

[andrewfagerheim]

20 April 2023: My Notes

Recap of the last two weeks:

• made (and debugged) a function for drho/dz and a function for "normalized eddy variance" where NEV = (EV/(drho/dz)**2)**(1/2)
• another way to express this is isopycnal displacement in meters, or how much isopycnals are disrupted by internal waves
• calculated and plotted spice and EV of spice (not an easy normalization default)
• finished (?) poster for climate school research showcase

Things to be working on (unspecific timeline):

• apply to summer programs
• try normalizations for comparing spice between regions
• use Dhruv's function from glider paper to go from depth space to density space
• read Roullet paper and compare results. Maybe identify new boxes to look at?
• write function to add spice to xarray. Maybe write a new function to load boxes based on input of lat/lon? this should:
  • load points
  • convert to xarray
  • convert points to profile
  • interpolate pressures
  • add spice
  • map box
• make functions for making standard plots quickly. These won't have all the precision you would want for a poster, but will help quickly see what's going on in a new box for example. Want to be able to make:
  • 4 panels from CCS poster (density, isopycnal displacement, spice, spice variance)
  • EKE with box displayed
  • T-S diagram

[andrewfagerheim]

Closing this and opening a new one for the summer