Closed kkappler closed 1 year ago
Extra Items
Once that is integrated, review whether to make tfk an input, or to give tfk a process() method, that basically is the same as process_mth5.
Merging #242 (afc0e5f) into main (d9932f0) will increase coverage by
1.38%
. The diff coverage is89.71%
.
@@ Coverage Diff @@
## main #242 +/- ##
==========================================
+ Coverage 77.53% 78.92% +1.38%
==========================================
Files 99 103 +4
Lines 5493 5798 +305
==========================================
+ Hits 4259 4576 +317
+ Misses 1234 1222 -12
Impacted Files | Coverage Δ | |
---|---|---|
aurora/config/config_creator.py | 90.62% <ø> (-1.27%) |
:arrow_down: |
aurora/config/metadata/decimation_level.py | 81.17% <ø> (ø) |
|
aurora/pipelines/transfer_function_helpers.py | 87.17% <ø> (-1.46%) |
:arrow_down: |
aurora/time_series/frequency_band.py | 68.11% <ø> (-0.46%) |
:arrow_down: |
setup.py | 0.00% <ø> (ø) |
|
aurora/pipelines/time_series_helpers.py | 75.18% <50.00%> (-0.39%) |
:arrow_down: |
...ra/test_utils/synthetic/make_processing_configs.py | 75.55% <51.85%> (+5.18%) |
:arrow_up: |
aurora/sandbox/mth5_channel_summary_helpers.py | 82.75% <60.00%> (-13.67%) |
:arrow_down: |
aurora/general_helper_functions.py | 61.29% <71.42%> (+1.29%) |
:arrow_up: |
aurora/test_utils/parkfield/make_parkfield_mth5.py | 80.85% <75.00%> (-0.24%) |
:arrow_down: |
... and 24 more |
... and 2 files with indirect coverage changes
Help us with your feedback. Take ten seconds to tell us how you rate us. Have a feature suggestion? Share it here.
ProcessingSummary Data Frame exists, it is built by TransferFunctionKernel (TFK)
Next steps:
[x] Modify process_mth5 to use tfk. In future, it may be passed as an argument, instead of config, and dataset, but probably best to keep passing config and tfk dataset separately (like it is now) and then TFK gets built as first step in process_mth5. This will leave the existing ipynbs as backward compatible.
[x] Move processing_config.validate_processing(tfk_dataset) into TFK
[x] move initialize_config method into TFK and out of process_mth5
[\] Modify tfk_dataset.initialize_dataframe_for_processing(mth5_objs) so that RunTS is not packed into df, only the run reference and the mth5 obj ** The way things are now, looping on decimation level, there is no clean way to loop over run without intermediate storage of decimated time series, otherwise we will need to redecimate the data for the 1st, 2nd, 3rd, and Nth decimation level all the way from zero. Better to await implementation of FC and or dec level data structures in mth5.
[\] Review the processing main loop; It should be done in a way that is groupby [survey, station, run, start, end], that way, every individual RunTS will be loaded only the one time ** See previous point - without storage we will take a serious performance hit in compute cycles.
[X] Fix all tests so that they are passing with this new method
[\] Modify all ipynb so that they are passing with this new method