The way the matlab code was setup, transfer functions are computed one at a time by looping regression over the "output" channels.
Aurora follows this pattern as well.
The regression however, cleans the data with an iterative M-estimator and the estimator class has that cleaned data, we just don't store it, but we could. If we did store the cleaned output channels after each individual output channel regression step, this would have a couple of implications:
We could difference the cleaned data from the original and get a full NCM
The cleaned data could be used to compute a cleaned SDM
These two entities could be used to solve both #329, and #87.
Also it is possible that we could then apply a general cleaned covariance approach to the TF.
Since the TF is just a ratio (of sums (of products)) of cross powers, the appropriate selection of these indices from the SDM should yield an equivalent TF to the one we have now.
This would also open the door to other data products (for example canonical coherences) that we could derive from the SDM.
To see if this makes sense, here is the test to do:
[ ] Test Proof of Concept
Inside process_transfer_functions (which currently lives in transfer_function_helpers.py) there are two lines:
These could be modified so that we access regression_estimator.Yc -- the cleaned data.
This cleaned data could be used to overwrite the input data
(this should be OK as the data are discarded after this step but could also make a copy and store in a cleaned_data vector)
Once all the TFs are estimated, we will have cleaned data for all output channels.
But NOTE that the horizontal magnetic channels will not have been cleaned!
This is the difference between the SS, RR SDM, and the MMT SDM, and the TFs yielded from that SDM may not be as trustworthy ... its hard to say how different they would be, the X (local horizontal magnetic) and RR (remote horizontal magnetic) channels do not appear to be modified in any way during the regression_estimator.estimate() process ...
The way the matlab code was setup, transfer functions are computed one at a time by looping regression over the "output" channels. Aurora follows this pattern as well.
The regression however, cleans the data with an iterative M-estimator and the estimator class has that cleaned data, we just don't store it, but we could. If we did store the cleaned output channels after each individual output channel regression step, this would have a couple of implications:
These two entities could be used to solve both #329, and #87.
Also it is possible that we could then apply a general cleaned covariance approach to the TF.
Since the TF is just a ratio (of sums (of products)) of cross powers, the appropriate selection of these indices from the SDM should yield an equivalent TF to the one we have now.
This would also open the door to other data products (for example canonical coherences) that we could derive from the SDM.
To see if this makes sense, here is the test to do:
process_transfer_functions(which currently lives in transfer_function_helpers.py) there are two lines:These could be modified so that we access
regression_estimator.Yc-- the cleaned data.cleaned_datavector)