The motivation behind the changes here are that the original data gathering setup used by Pyatoa was very abstract, opaque, and unnecessarily rigid, e.g.,, building path strings out of various components of filenames and internal attributes, or hiding preprocessing functions behind abstract function calls.
Changes were made to make tasks in Pyatoa more explicit rather than implicit, hopefully making it easier to use and manipulate both as a direct tool, and in larger workflow tools.
Finally, data gathering has been completely removed from this package, and the new recommend approach is to use PySEP to perform all data gathering once-and-for-all, including one time tasks like instrument removal. The resulting SAC files can then be read in with ObsPy and directly fed into the Manager class for misfit quantification.
What does this PR do?
Simplifies the User interface with the package by reducing arbitrary complexity in the data gathering routines.
Removes a lot of opaque abstraction that was initially written into the package to keep things "on rails" but ended up being overly restrictive.
Improves some internal features like flow_multiband and Config setting underlying Pyflex and Pyadjoint Configs.
CHANGELOG:
Removed pyatoa.core.gatherer.Gatherer class from package entirely, all
data gathering capabilities have been migrated to PySEP, Pyatoa will now only
accept input data as already-defined ObsPy objects
Removed Gatherer-related tests and documentation from package
Removed paths attribute from pyatoa.core.config.Config and all
references to the paths attribute throughout the package as these were only
accessed by the now removed Gatherer class
Changed Pyflex and Pyadjoint configuration building procedure in
pyatoa.core.config.Config as it was previously abstracted behind a few
unncessary functions. Config now accepts parameters pyflex_parameters
and pyadjoint_parameters (dictionaries) that overwrite default Config
parameters in the underlying Config objects
Changed pyatoa.core.manager.Manager.write() to write_to_dataset to be
clearer in explaning it's role
Exposed the default preprocessing procedures directly in the
Manager.preprocess function, rather than having it hidden behind a
function call to a utility script. Users who want to overwrite the
preprocessing need only skip the call to preprocess and perform their own
tasks on the internally defined st_obs and st_syn attributes.
Removed pyatoa.core.manager.Manager's ability to save to ASDFDataSet mid
workflow (i.e., during window and measure). Manager must now use the
write_to_dataset function if it wants to save data to an ASDFDataSet
Removed the pyatoa/plugins directory which only contained the pyflex
preset dictionaries. These were not very flexible, instead they have been
converted to a docs page for easier accessibility.
Created Docs page for Pyflex presets that can be copy-pasted into misfit
quantification routines
Added a plt.close('all') to the end of the Manager's plot routine as
as a final precaution against leaving an excessive number of Matplotlib
figures open (maybe related to #36?)
Overhauled pyatoa.core.manager.Manager.flow_multiband to mimic behavior
the standard behavior of Manager.flow, that is: return internal attributes
windows and adjsrcs which are component-wise dictionaries that each
contain Pyflex Windows and Pyadjoint AdjointSource objects, respectively.
Previously this function returned dictionaries of dictionaries which needed
to be further manipulated, now the function averages all adjoint sources
from all period bands, and also collects all windows.
Removes utils.images and migrates them to SeisFlows as these utilities
were only required for the now-removed Pyaflowa interaction class (#37)
Adjusted and fixed tests based on all the above changes.
PR Checklist
[x] devel base branch selected?
[x] This PR is not directly related to an existing issue (which has no PR yet).
[x] All tests still pass.
[x] Any new features or fixed regressions covered by new tests.
[x] Any new or changed features have been fully documented.
[x] Significant changes have been added to CHANGELOG.md.
[ ] First time contributors have added your name to CONTRIBUTORS.txt .
Why was it initiated? Any relevant Issues?
The motivation behind the changes here are that the original data gathering setup used by Pyatoa was very abstract, opaque, and unnecessarily rigid, e.g.,, building path strings out of various components of filenames and internal attributes, or hiding preprocessing functions behind abstract function calls.
Changes were made to make tasks in Pyatoa more explicit rather than implicit, hopefully making it easier to use and manipulate both as a direct tool, and in larger workflow tools.
Finally, data gathering has been completely removed from this package, and the new recommend approach is to use PySEP to perform all data gathering once-and-for-all, including one time tasks like instrument removal. The resulting SAC files can then be read in with ObsPy and directly fed into the Manager class for misfit quantification.
What does this PR do?
flow_multibandand Config setting underlying Pyflex and Pyadjoint Configs.CHANGELOG:
pyatoa.core.gatherer.Gathererclass from package entirely, all data gathering capabilities have been migrated to PySEP, Pyatoa will now only accept input data as already-defined ObsPy objectspathsattribute frompyatoa.core.config.Configand all references to the paths attribute throughout the package as these were only accessed by the now removedGathererclasspyatoa.core.config.Configas it was previously abstracted behind a few unncessary functions.Confignow accepts parameterspyflex_parametersandpyadjoint_parameters(dictionaries) that overwrite default Config parameters in the underlying Config objectspyatoa.core.manager.Manager.write()towrite_to_datasetto be clearer in explaning it's roleManager.preprocessfunction, rather than having it hidden behind a function call to a utility script. Users who want to overwrite thepreprocessing need only skip the call to preprocess and perform their own tasks on the internally defined
st_obsandst_synattributes.pyatoa.core.manager.Manager's ability to save to ASDFDataSet mid workflow (i.e., during window and measure). Manager must now use thewrite_to_datasetfunction if it wants to save data to an ASDFDataSetpyatoa/pluginsdirectory which only contained the pyflex preset dictionaries. These were not very flexible, instead they have been converted to a docs page for easier accessibility.plt.close('all')to the end of the Manager's plot routine as as a final precaution against leaving an excessive number of Matplotlib figures open (maybe related to #36?)pyatoa.core.manager.Manager.flow_multibandto mimic behavior the standard behavior ofManager.flow, that is: return internal attributeswindowsandadjsrcswhich are component-wise dictionaries that each contain Pyflex Windows and Pyadjoint AdjointSource objects, respectively. Previously this function returned dictionaries of dictionaries which needed to be further manipulated, now the function averages all adjoint sources from all period bands, and also collects all windows.utils.imagesand migrates them to SeisFlows as these utilities were only required for the now-removed Pyaflowa interaction class (#37)PR Checklist
develbase branch selected?CHANGELOG.md.