This maybe a bug or something the processor needs to figure out before running process_mth5 but here's a scenario that happens.
Typical instruments have multiple sample rates and different ways of saving those data to disk. Occasionally, there is a hiccup in saving and maybe one channel gets corrupt for that run. So you may have run 1 with all 5 channels, run 2 has 4 channels and run 3 has all 5 channels again.
If you try to process these 3 runs together an error will occur because the spectrograms will be different shapes for the run with the missing channel. That is because drop_nans wipes out the missing channel and all the data that in the other channels for the missing data. So if you have the missing channel is an output channel like ex then ey and hz will also be wiped out over the missing time period of ex. Then when you go to estimate weights there is a mismatch in shapes between the input and output channels because output will contain info for all 3 runs.
Because nans are taken care of in the loop on line 252 in pipelines\transfer_function_helpers.py | process_transfer_function call in line 244 pipelines\transfer_function_helpers.py | process_transfer_function | X, Y, RR = drop_nans(X, Y, RR) might be superfluous. When I comment out that line you can process the 3 runs, now you aren't exactly processing the 3 runs you are processing the 2 that have all the same channels and skipping the odd ball. There probably isn't a better way of handling this programmatically unless because drop_nans will drop all channels for which there are nans in the xr.Dataset.
Maybe there could be a check in RunSummary.check_runs_are_valid that identifies runs that are different from the rest. I guess you need at least 3 runs to do the check to find the odd one out. And just provide a warning to the user? I'll create a PR for this.
This maybe a bug or something the processor needs to figure out before running
process_mth5but here's a scenario that happens.Typical instruments have multiple sample rates and different ways of saving those data to disk. Occasionally, there is a hiccup in saving and maybe one channel gets corrupt for that run. So you may have run 1 with all 5 channels, run 2 has 4 channels and run 3 has all 5 channels again.
If you try to process these 3 runs together an error will occur because the spectrograms will be different shapes for the run with the missing channel. That is because
drop_nanswipes out the missing channel and all the data that in the other channels for the missing data. So if you have the missing channel is anoutputchannel likeextheneyandhzwill also be wiped out over the missing time period ofex. Then when you go to estimate weights there is a mismatch in shapes between theinputandoutputchannels becauseoutputwill contain info for all 3 runs.Because
nans are taken care of in the loop on line 252 inpipelines\transfer_function_helpers.py | process_transfer_functioncall in line 244pipelines\transfer_function_helpers.py | process_transfer_function | X, Y, RR = drop_nans(X, Y, RR)might be superfluous. When I comment out that line you can process the 3 runs, now you aren't exactly processing the 3 runs you are processing the 2 that have all the same channels and skipping the odd ball. There probably isn't a better way of handling this programmatically unless becausedrop_nanswill drop all channels for which there are nans in the xr.Dataset.Maybe there could be a check in
RunSummary.check_runs_are_validthat identifies runs that are different from the rest. I guess you need at least 3 runs to do the check to find the odd one out. And just provide a warning to the user? I'll create a PR for this.