Open kkappler opened 2 years ago
Notes from a chat with Gary
This is really a segment rejection method
Standard Multiple Coherence: -R2 is just the fraction of the E field signal power that is Fit -Use the residuals to weight the data --
Could also use magnetic vs magnetic from remote station. This would just identify segments, it is done before we do the processing pipeline.
You use a much wider band than your estimation band to do these estimations.
The FORTRAN implementation is aimed at single station data where there is no remote.
One way to implement this:
Note that in some implementations, a direct ExEx is multiplied into the "diminished estimate" and the sqrt of their product is taken. In either case, 0 < [<ExEx>_1 / <ExEx>_2] < 1.0 and ranking these and selecting a precentile cut from below will increase SNR in the dataset fed to RME.
The task of extracting features (cross-powers and impedance estimates per time-window) should be treated as separate tasks from the coherence sorting. These are necessary, but not sufficient. The features may also find use for other weighting or data QC/viz schemes in future.
Go forward plan:
process_tf_decimation_level
function.Probably we can make STFT into a class, and then coherence_weights can be a method of that class.
Note that issue #316 or a similar solution will likely need to be addressed. The SigMT version of this seems similar to the method that Gary described.
Consider these two distributions of simple coherence from the same band:
It turs out that the first one seems like a reasonable candidate for jaackknife coherence, but the second one has some fairly non-intuitive results, in that the ensemble that gives the lowest partial coherence is not necesarily one with particularly poor simple coherence. Something to think about ...
Probably, application of jackknife prematurely not a good idea ... probably should apply simple and or multiple coherence before jackknife.
A basic coherence sorting has been implemented. This improves the highest frequency estimates at for example IAK34, but there are still some issues with phase, and a few bad periods in aurora that are not showing as bad in EMTF. The existing method is a sort-of jackknife that iteratively removes a segment of data and recomputes the coherence of the remaining ensembles, either local of remote. The coherence is only between two channels though (not multiple coherence)
To do multiple coherence we should make:
get_band_for_tf_estimate
that extracts the coherence bandThere are several issues here:
References:
Egbert, Gary D. New approaches to estimation of magnetotelluric parameters. Final technical report, 1 August 1989--31 July 1991. No. DOE/ER/14057-2. Oregon State Univ., Corvallis, OR (United States). Coll. of Oceanography, 1991a.
Egbert, Gary D., and Dean W. Livelybrooks. "Single station magnetotelluric impedance estimation: Coherence weighting and the regression M-estimate." Geophysics 61.4 (1996): 964-970.
Jones, Alan G., and Hartmut Jödicke. "Magnetotelluric transfer function estimation improvement by a coherence-based rejection technique." SEG Technical Program Expanded Abstracts 1984. Society of Exploration Geophysicists, 1984. 51-55.
Smirnov, M. Yu. "Magnetotelluric data processing with a robust statistical procedure having a high breakdown point." Geophysical Journal International 152.1 (2003): 1-7.