Temporal data gaps (Axis 55 - 2018/2019)

[kurtisanstey]

Using appropriate averaging and weighting. Trying to automate it so I don't have to do it manually every time I want these years or seasons.

[kurtisanstey]

Doing the windows individually and averaging them myself. This may take a bit, but I'll save the final spectra from each window so I don't have to do it again.

[kurtisanstey]

@jklymak

It's working! I've checked for other years, but this example (Axis 55 kHz 2017) has only one small data gap, so the new process PSD can be reasonably compared with the old method that used interpolated data. I do use a slightly longer nperseg window in the new process, and the interpolated gap is of course absent, but it looks good.

Rough velocities to show data gap:

Old PSD from interpolated data:

New PSD from weighted average process:

I then expanded the segment by segment data approach to the rest of my analysis processes (just finishing up applying it to the rotary spectra), to reduce the amount of overall interpolation that existed in my code. Everything is working nicely:

[kurtisanstey]

@jklymak

• To supplement the single gap comparison, above. An additional check for Axis 55 kHz in 2018 with many data gaps, some large, where previously analysis was only possible with very long interpolation periods (weeks).
• New plots will be a bit different, without the large interpolated portions and updated averaging for spectra, but I think it's a good comparison.
• Sorry for so many plots, but I want this to be pretty comprehensive to show the new process is working properly.

  ---

  Rough velocities to show gaps:

  ---

  PSD with interpolation over gaps (old process): PSD with new process (very low frequencies at -913m are reduced, likely since no interpolation):

  ---

  Rotary spectra with interpolation over gaps (old process): Rotary spectra with new process (same low frequency changes as PSD):

  ---

  Spectrograms with interpolation over gaps (old process): Spectrograms with new process (data gaps are emphasised by the windowing and overlap now that the gaps are NOT interpolated, and I used a bit more averaging):

  ---

  Velocities with interpolation over gaps (old process): Velocities with new process (including filtered and residual):

[kurtisanstey]

These processes are working.