adjtomo / pysep

Seismogram Extraction and Processing: Seismic data retrieval and record sections
https://pysep.readthedocs.io
MIT License
29 stars 15 forks source link

Pysep ignoring maxdistance parameter in pysep_config.yaml? #110

Closed ammcpherson closed 1 year ago

ammcpherson commented 1 year ago

Hello,

Whilst downloading data for an event, I noticed that the log message on screen said that the maximum distance used was the default 2000 km ([2023-07-05 11:43:43] - pysep - INFO: 0 traces outside distance bounds [0, 20000.0]km ) instead of the 300 km that I requested. I am using the latest version of Pysep on the devel branch, and have attached my pysep_config.yaml file.

As a side note, once the data was downloaded, Pysep also did not write a weight.dat file. I am unsure if this is related to the obscene amount of data and potential timeout, or if I need to open a new issue for this.

Thanks! pysep_config.yaml.gz

bch0w commented 1 year ago

Hi @ammcpherson, sorry about that, some time back I changed the parameter maxdistance -> maxdistance_km to be more explicit about the expected units (https://github.com/adjtomo/pysep/issues/104), so Pysep is ignoring the value you provide for maxdistance.

I can add a small warning message when loading the config file so that future User's are notified when they provide an input parameter that is not expected/used by PySEP.

Additionally, to double check your parameter file, you can run pysep -W to create a template file, which should provide you with the up-to-date argument names.

Related to the 'weight.dat' file, I think PySEP no longer writes out weight.dat explicitely, but instead writes weights_dist, weights_az and weights_code, which all sort the weight file by different categories.

https://github.com/adjtomo/pysep/blob/devel/pysep/pysep.py#L1539-L1543

This is sort of general as I wasn't sure exactly how CAP or MTUQ expected this file, happy to work on finding the optimal file naming/sorting scheme to output by default.

ammcpherson commented 1 year ago

Thanks for the explanation, @bch0w. I updated my config file and I'm now getting the amount of data that I expect.

As far as the weights file, old Pysep always output the weight file sorted by distance, which matched the default sorting of data/synthetics for CAP and MTUQ. However, that default sorting was lost in the refactor, and I think it was alphabetical by station code. This had not caused any problems for MTUQ.

Is there a specific flag that I can use in the config file to write out any of these new weight files? None of them are being written out by default, and having any weight file at all is important for MTUQ.

Thanks!

bch0w commented 1 year ago

Glad that worked, @ammcpherson!

Yes seems like I dropped writing weights files by default at some point. I also misspoke in my previous comment, PySEP will still write weights as weights.dat, but you can choose how the lines are sorted. If you add the following to your config file, PySEP will write out the default expected files, as well as a weights.dat file sorted by distance.

write_files: "inv,event,stream,sac,config_file,station_list,weights_dist"

Here is the docstring that defines the acceptable values for the write_files parameter: https://github.com/adjtomo/pysep/blob/devel/pysep/pysep.py#L399-L422

and the default write_files behavior is defined here: https://github.com/adjtomo/pysep/blob/devel/pysep/pysep.py#L74

Note that currently PySEP will only write out one weights.dat file, so you can only choose one sorting option from: weights_code, weights_dist, weights_az

ammcpherson commented 1 year ago

Thanks, @bch0w, I'm getting all the things I need from Pysep again!