SUMMER 2022: Roadmap/Project Flow

[andrewfagerheim]

Repo Organization & Project Outline

Daily Tasks:

• Update and Resolve #2
• Update and Reference #3
• Update and Refer to #4
• Update and Reference Google Calendar

Project Flow:

1. Create synthetic example of a signal by combining sin functions as a first exercise
2. Work with glider data and try to reproduce spectral analysis
3. Analyze Argo data in this region and hope it displays layering somewhat similar to the glider results
4. Slowly broaden scope to include more geographic regions once the above code seems to work

[jbusecke]

😍

[andrewfagerheim]

2 June 2022: Intro Meeting w @dhruvbalwada

BROAD CONCEPT: Physical Oceanography

• Tracers like temp, salinity allow us to track water masses and how they move spatially over time
• Temperature is relatively constant over time (scale of a few years) but dissipates as the distance from the initial formation site increases, salinity is also relatively constant over this timescale but can also be influenced by precipitation/evaporation, oxygen and nutrients are more complex because they are dependent on biological interactions
• These constant properties are maintained along the path that the water travels along, which is primarily along isopycnals (surfaces of constant density)

SPECIFIC DATASET: Argo Floats

• Almost 4,000 active worldwide, many were purchased by the U.S. but it’s an international effort with broad geographic distribution
• Passive lateral transportation (driven by currents) but standardized depth movement - move to 1,000m below the surface, remain there for 10 days, move to 2,000 below the surface, then gather data on their way up
• Newer floats have resolution with respect to depth of ~2m, older ones ~10m

BASICS OF THIS PROJECT

• Determine statistic to represent “structure” of depth of water column: Spectral analysis uses a Fourier transformation to break one signal into its component parts and graph the ratio between amplitude and wavelength. Plotting this against depth would allow the depth where the slope changes (i.e. where “busier” areas with more structure give way to smoother/quieter areas). This reduces the data collected over a given depth to be condensed into one number and then plotted on a map
• Map this statistic across latitude and longitude to see what broader trends it might connect to

INFORMATION FLOW

• Create maps that show how the “structure” of the surface 2,000m of ocean changes with respect to latitude and longitude
• Develop parameters for climate models, information that is too localized for a climate model to include, so instead assumptions are built into it
• Gain a better understanding of what resolution is necessary for climate models

[andrewfagerheim]

3 June 2022: Logistics Meeting w @dhruvbalwada @jbusecke

TODO

• See #2 and #3 dated June 3 for action items and references that came from this meeting
• Use argopy as a resource when using the Argo dataset
• See first comment in this issue for project roadmap that came from this meeting

ARGO FLOAT COMMS

• Resolution is function of what information is actually sent back, limitation of electronics and communication ability
• Uses satellite for communication, Argo comes from argonauts, ARGOS satellite constellation, speed/amount that can be communicated is dependent on which satellite is being used
• Does the equipment sample more frequently? (Based on rates) Sampling at a high rate and then averaging avoids aliasing, snapshots will run into this problem

[andrewfagerheim]

6 June 2022: Meeting w @jbusecke

DISCUSSION:

• See #2 and #3 dated June 6 for action items and references that came from this meeting
• Focused on gaining access to Gyre and setting up Dhruv's notebook. This involved using the terminal to generate an SSH key and connecting it to GitHub, creating a new environment that loads the required Python packages, and updating that environment
• Will need to follow a similar process to connect GitHub and the Pangeo Cloud once I have access

NEXT STEPS:

• Ensure all packages run correctly (notebook needs xrft)
• Start editing Dhruv's notebook (create a copy first) to start understanding how the data is formatted/analyzed
• Learn how to push (?) updates from Gyre to GitHub

WORK DONE AFTER MEETING:

• Updated conda from version 4.8.4 to 4.13.0 as prompted in the terminal

• Added the xrft package and got (almost) everything to run correctly, yet it still shows this error for the line screenshotted here @jbusecke any idea what this is referring to?

• The above screenshot is a warning NOT an error which means that the code can currently run as intended. This one is stating that one of the features used may not be available in the future.

[jbusecke]

8 June 2022: Meeting w @andrewfagerheim

DISCUSSION:

• Reviewed updates to Dhruv's notebook including looking at profiles in multiple locations, adding salinity plots next to Dhruv's temperature ones, plotting the glider path on a map
• Demonstrated how to commit and push updates to the repo, see notes dated today in #3

POTENTIAL NEXT STEPS:

• Reproduce profile and sections for Argo float
• Explore floats in different regions (start close to Dhruvs glider locations)
• Familiarize yourself with vertical resolution

ACTION ITEMS:

• [x] Use argopy dashboard and fetcher to see profiles of temperature and salinity for individual floats. Select ones that may show interesting trends or be in relevant locations.
• [x] Write code that produces similar profile and section plots to Dhruv's for one float
• [x] Expand this code to loop through all floats provided in a list to produce the relevant plots for each
• [x] Examine how the resolution of depth impacts the plots for floats in similar areas

[andrewfagerheim]

10 June 2022: Progress Update after June 8 Meeting w @jbusecke

PROGRESS:

• Selected 8 floats from different geographic regions that show interesting trends including: clear striation of temperature/salinity over depth, unexpected temperature/salinity patterns (as you move deeper it gets cooler, then warmer, then cooler again), uniformity across all areas except the very surface water
• Mapped the paths of these floats to see their geographic extent
• Able to commit and push updates into GitHub, use basic bash commands

NEXT GOALS:

• [x] Use the interpretation method to define pressure levels (see guide)
• [x] Use the bin method to define pressure levels (see guide)
• [x] Create profiles of temperature/salinity using each of the above methods. Are there significant differences in the resolution of floats? Does one method work better for higher/lower resolution data?
• [x] Create temperature/salinity anomaly plots like in Dhruv's notebook

[andrewfagerheim]

10 June 2022: Meeting w @jbusecke

NEXT STEPS:

• [x] Define a function that reads in a float ID number, fetches the data, and returns 4-5 plots
• [x] Use this function to find and record floats that have interesting/significant profiles

QUESTION: What plots should I return? The options we talked about include:

• Map of float path
• Pressure vs salinity profile
• Salinity colormesh
• Salinity colormesh anomaly, overall
• Salinity colormesh anomaly, vertical
• All of the above but for temperature

QUESTION: What things am I looking for in an "interesting" float?

[andrewfagerheim]

13 June 2022: Meeting w @jbusecke @dhruvbalwada

RANDOM NOTES:

• Sampling frequency is 2dbar (roughly 2m resolution) for this plot, found using: afloat.PRES.diff('N_POINTS').plot.hist(bins=np.linspace(0,5,100))
• Plot density as well, will need to calculate, use a TEOS10 package (SEE LINK BELOW) to run those calculations

GOALS: Short Term (by Wednesday)

• [x] Tweak function: remove vertical anomaly plot, include temperature, salinity, and density plots under one function
• [x] Use function to examine floats near the glider. Do they display similar patterns to Dhruv's notebook?

GOALS: Long Term (coming weeks)

• Define method to remove mixed layer more quantitatively than visual inspection
• Remove "expected/universal" patterns using climatology package? or slow moving mean?
• Analyze the results using spectral analysis? or another method?

[jbusecke]

DEOS10

This is actually TEOS10. Here is the python package

[andrewfagerheim]

15 June 2022: First Focusing Session

Questions/Feedback

• Wanted to know what conservative temperature means from Dhruv's plot
• Wanted to know specific outcomes/purpose of my project, unique contribution compared to many other existing analysis of ocean structures
• Comments about how there is a lot of data to sift through, asked about how using all the floats is beneficial

[andrewfagerheim]

16 June 2022

Notes on selected floats in the Southern Ocean:

• _[index in floatli] float ID: my notes
• [1] 1902218: one distinct layer (particularly in salinity), almost looks like it follows a sin wave
• [2] 6902777: good example of seasonality effects (particularly in temperature)
• [5] 7900510: "break" in colormesh around profile 30
• [8] 1901837: may show layering, hard to tell with seasonality
• [9] 1902030: may show layering in anomaly although the profiles look smooth
• [11] 3901949: we only have ~80 profiles for this float but Euro Dashboard has ~160??
• [13] 3901963: we only have ~40 profiles for this float but Euro Dashboard has ~160??
• [14] 7900517: strong signal near the surface, then uniform, discontinuity in anomaly around profile 45

Other comments/issues:

• The difference in resolution is very noticeable. Many of these floats have ~50 profiles compared to 500 for the glider. An order of magnitude difference is significant (float_li indexes 0, 3, 4, 6, 7)
• Temperature colormesh sometimes appears as an anomaly plot (float_li indexes 7, 11, 13, 14)
• Some floats seem to have way less data than Euro Dashboard

[andrewfagerheim]

• Wanted to know what conservative temperature means from Dhruv's plot

See quick notes in https://github.com/ocean-transport/argo-intern/issues/4#issuecomment-1158989406

It seems like potential temperature is a measurement of temperature adjusted for adiabatic pressure change and conservative temperature is a measurement of enthalpy/internal energy.

[andrewfagerheim]

21 June 2022: Meeting w @dhruvbalwada

NOTES ON float_plot:

• For fewer profiles issue, is that because argopy only returns corrected data while dashboard shows everything?
• Use QC data instead? Is this the only part of the argopy dataset that's quality adjusted?
• Just use matplotlib to avoid the interpolated pressure issue altogether? xarray uses this "under the hood" anyway
• To speed this up, could add a loop where it tries to retrieve the data from a downloaded file and if it can't find it, it downloads the file and runs off this NOT the argopy fetcher

ARGO SPECIFICS:

• Argo is the middle/compromise of ocean sampling methods
  • Ships provide very limited spatial extent, but can collect samples and therefore measure many more properties
  • Argo floats have decent lateral resolution and good spatial extent, good vertical resolution across depth
  • Gliders have excellent lateral resolution but very limited extent

THIS PROJECT:

• Paper looks at horizontal structure, essentially concludes that you can't use Argo for this because to find multiple floats with structures smaller than 100km, would need timescale of ~10 days
• Using Argo to construct global analysis of vertical structure is unique, Argo generally has good sampling across depth
• This is why it makes sense to look at vertical structure, although it adds another level of interpretation because we are studying the vertical layers produced by horizontal stirring
• Also, "traditionally" the conceptual picture of stirring is based on models that don't have vertical axes, therefore new contribution is also the statistical representation of vertical structures

SPECTRAL ANALYSIS:

• Breaking up a busy signal into different wave modes using Fourier transformation; making a complex function by adding sin/cos functions with varying amplitude, wavelength, and phase
• Graph amplitude vs wavelength for each curve to create a line with intercept and slope (or potentially multiple slopes)
• DRAWBACKS
  • Assumes the signal follows a periodic function when it actually may not
  • Non-local, ie lose spatial variability if there are different patterns in different sections of the profile

FILTERING ANALYSIS:

• Define a "window" width to examine the data by, average all values within that window
• Changing the window width allows you to examine different wave modes
• BENEFITS: doesn't assume periodic function, better captures spatial changes

SHORT-TERM GOALS (by Friday):

• [x] Create slides for focusing session on Wednesday
• [x] Create notebook that combines a few sin curves, change amp/wavelength and see how this visually effects the function
• [x] Try spectral analysis on the above function and troubleshoot
• [x] Try filtering analysis on the above function and troubleshoot (unclear if this will happen this week)
• [ ] Try tips to speed up float_plot (this is definitely not a top priority)

LONG-TERM GOALS:

• Divide map into boxes of dozens/hundred? km and conduct spectral analysis on each profile within the box
• Create an average spectral analysis curve for each box
• Map properties of these curves (slope, intercept, intersection) across a global map

[andrewfagerheim]

22 June 2022: Second Focusing Session

Questions/Feedback:

• Asked about data reliability; how to identify/filter out erroneous data (sensor malfunction, etc), how to make sure that non-erroneous anomalies are not rejected
• Asked a lot about climatology; why the effects of seasons need to be removed, how this would happen, what dataset it's compared to
• Wondered if climatology even needs to be considered if mixed layer is removed

[jbusecke]

Wondered if climatology even needs to be considered if mixed layer is removed

I would say absolutely. While the mixed layer is the part of the ocean that is in direct contact with the atmosphere, the thermocline beneath it is still very seasonal (affected by lateral processes and vertical mixing).

[andrewfagerheim]

24 June 2022: Meeting w @dhruvbalwada

XARRAY ATTEMPT:

• define new variables:
  • points: gridsize, number of points that will form the x axis
  • modes: number of waves that comprise the final function
• y_li shouldn't be a list because this is just a list of numpy arrays which is unhelpful; instead make a np array with dimension (points, modes)
• data array should contain the data being considered (y_li/y_funct), dims: points and modes, coords: points

NOTES ON SPECTRAL ANALYSIS:

• lowest wavenumber: 2pi/L(domain); highest wavenumber (Nyquist frequency): 2pi/2del(x) (ie grid size)
• See #3 on THEOREMS for background on why spectral curve is k vs A^2

FILTERING ATTEMPT:

• the technique I used is the "top hat" method with coarsening because I averaged across the width of a bin, reducing the total number of datapoints

NOTES ON FILTERING:

• some filters will create a more coarse curve (top hat) and others will be more smooth (Gaussian)
• See #3 on FILTERS for information on convolution and the scipy function

SHORT-TERM GOALS:

• [x] Create spectral curves with slopes [0, -1, -2, -3] for a combined function with lots of sin functions
• [x] Try filtering technique with scipy.signal.convolve and different methods
• [ ] Readings (see above)

LONG-TERM GOALS:

• Apply spectral analysis code to Argo profiles
• Apply filtering techniques to Argo profiles: use multiple filtering methods (more coarse and more smooth), calculate variance for each, make similar plot to spectral analysis with variance

[andrewfagerheim]

30 June 2022: Meeting w @dhruvbalwada

SPECTRAL NOTES:

• Needed to fix a number of existing issues:
  • want wavenumbers to be spaced logarithmically
  • want amplitudes to be selected based on wavelength and slope a**2=ck**-m --> log(a**2)=log(c)-m*log(a)
  • ∴ k = np.logspace(0,n,modes)
  • ∴ a = k**(-m/2)
• Need to use axes spaced logarithmically
• In the sin function, multiply x by n*np.pi
• To check that power spectrum plot looks good, also plot plt.plot(k_li/(2*np.pi),10*a_li**2)

FILTERING NOTES:

• "Points" on a boxcar plot defines the size of the window, so this should be much smaller than "points" used to define the function
• Gaussian window creates a smoother signal because it goes to zero on the tails quicker than the other two
• Sigma for Gaussian and tau for exponential set the window width where higher numbers produce smoother plots
• Still a bit unclear how boxcar works, particularly why it doesn't look much smoother even as width is increased significantly

SHORT-TERM GOALS:

• [x] Define a function that creates a signal based on given parameters (points, modes, slope, x-axis length)
• [x] Use this function to plot the power spectrum of signals with slopes [0,-4]
• [x] Define a function that creates a Gaussian filtered function based on parameters (signal, sigma)
• [x] Use this function to filter the signal with slope -2 with a range of sigma values, plot power spectrum for each of these sigmas (make sure to chop off boundary before)

[andrewfagerheim]

1 July 2022: Meeting w @dhruvbalwada @jbusecke

sigma dependent on grid spacing

calculate spectra of filtered signal

see where it falls off

represent sigma in terms of filter length scale, choose filter length scale, determine del(x), think of gridpoints in terms of x

give function filter scale, define length scale in terms of distance increment, how does sigma relate to this

xrft windowing? calculate spectra with window=True, window_correction? see how this impacts orange line

just pick Argo float and try this out now, or glider profile since that already has interesting structure

take filter off of signal itself to create anaomly plot, create variance plots too

read Wiki sites for variance and standard deviation, this will play into

NOTES PLEASE:

• notes

SHORT-TERM GOALS:

• [ ]

[andrewfagerheim]

8 July 2020: Meeting w @dhruvbalwada

FILTER SCALE:

• instead of defining sigma arbitrarily, define sigma in terms of the filter window length, where sigma=L_filter/dx/np.sqrt(12) and L_filter is a value passed to the function
• first wavenumber is 1/xmax and the power spectrums diverge just before L_filter at 1/L_filter

ASSUMPTIONS:

• assuming that filtering out the large-scale frequencies will also take out climatology [assuming var(T(L))~var(T(C)), var(T')>>var(T(C))]
• assuming that the extent of seasonality only penetrates to a certain depth, see how far this goes beneath the mixed layer

SHORT TERM:

• [x] update vertical structure notebook to include many more filter width sizes (is there a slope change like the power spectrum?)
• [x] load profiles from 10deg x 10deg box and try to apply filtering to it (pay attention to vertical resolution/binning, max depth, etc
• [ ] CATCH UP ON READINGS!!!

[andrewfagerheim]

13 July 2022: Meeting w @dhruvbalwada

QUESTIONS:

• How does spectral curve relate to variance plot for different filter lengths?
• How does argo.point2profile() choose how many N_LEVELS to return?
• How does argo.interp_std_levels work? (Is it linear interpolation?)

[andrewfagerheim]

Closing issue from this summer and opening a new one for the fall