search

PIC-IRIS / PH5

Library of PH5 clients, apis, and utilities
Other
15 stars 9 forks source link

[BUG] ph5toms: Data extraction inconsistent when number of input files is too large #500

Closed hrotman-pic closed 2 years ago

hrotman-pic commented 2 years ago

Describe the bug It appears that for some stations, the ability of ph5toms to extract data from some dates & times depends on the query parameters. I discovered this when I tried to extract all data for a station's deployment. I think this does not affect all stations in an experiment, but may require additional testing to determine if this bug is limited to a few stations per experiment or is relatively widespread.

I will include here examples from 2 stations (each from a different experiment), the queries, and msi output (some of the msi output is fairly long). The first example also includes duplication of the queries on web services; web services and PIC server ph5toms appear to operate the same. Both experiments used here can extract data from all stations.

Environment (please complete the following information):

To Reproduce (Examples)

Experiment 21-002

Station used: C7035 (id_s=7035)

  1. Entire deployment

    • PIC server: ph5toms -n master.ph5 -a 002 --station C7035 -o mseed_stnC7035_test2 -F MSEED

    • msi -G 9N* Source Last Sample Next Sample Gap Samples 9N_C7035GP1 2020,316,18:42:05.002000 2020,316,18:14:18.000000 -1667 1667003 9N_C7035__GP1 2020,317,18:14:18.046999 2020,317,18:14:18.000000 -0.047 47.999 9N_C7035GP1 2020,317,23:59:59.999998 2020,319,00:00:00.000000 1.0d 86399999 9N_C7035GP1 2020,320,00:00:00.046998 2020,320,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,321,00:00:00.046998 2020,321,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,322,00:00:00.046998 2020,322,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,323,00:00:00.046998 2020,323,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,324,00:00:00.046998 2020,324,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,324,23:59:59.999998 2020,326,00:00:00.000000 1.0d 86399999 9N_C7035GP1 2020,327,00:00:00.046998 2020,327,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,328,00:00:00.046998 2020,328,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,329,00:00:00.046998 2020,329,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,330,00:00:00.046998 2020,330,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,331,00:00:00.046998 2020,331,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,331,23:59:59.999998 2020,333,00:00:00.000000 1.0d 86399999 9N_C7035GP1 2020,334,00:00:00.046998 2020,334,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,335,00:00:00.046998 2020,335,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,336,00:00:00.046998 2020,336,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,337,00:00:00.046998 2020,337,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,338,00:00:00.046998 2020,338,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,339,00:00:00.046998 2020,339,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,340,00:00:00.046998 2020,340,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,340,23:59:59.999998 2020,342,00:00:00.000000 1.0d 86399999 9N_C7035GP2 2020,316,18:42:05.002000 2020,316,18:14:18.000000 -1667 1667003 9N_C7035GP2 2020,317,18:14:18.046999 2020,317,18:14:18.000000 -0.047 47.999 9N_C7035__GP2 2020,317,23:59:59.999998 2020,319,00:00:00.000000 1.0d 86399999 9N_C7035GP2 2020,320,00:00:00.046998 2020,320,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,321,00:00:00.046998 2020,321,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,322,00:00:00.046998 2020,322,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,323,00:00:00.046998 2020,323,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,324,00:00:00.046998 2020,324,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,324,23:59:59.999998 2020,326,00:00:00.000000 1.0d 86399999 9N_C7035GP2 2020,327,00:00:00.046998 2020,327,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,328,00:00:00.046998 2020,328,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,329,00:00:00.046998 2020,329,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,330,00:00:00.046998 2020,330,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,331,00:00:00.046998 2020,331,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,331,23:59:59.999998 2020,333,00:00:00.000000 1.0d 86399999 9N_C7035GP2 2020,334,00:00:00.046998 2020,334,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,335,00:00:00.046998 2020,335,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,336,00:00:00.046998 2020,336,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,337,00:00:00.046998 2020,337,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,338,00:00:00.046998 2020,338,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,339,00:00:00.046998 2020,339,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,340,00:00:00.046998 2020,340,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,340,23:59:59.999998 2020,342,00:00:00.000000 1.0d 86399999 9N_C7035__GPZ 2020,316,18:42:05.002000 2020,316,18:14:18.000000 -1667 1667003 9N_C7035GPZ 2020,317,18:14:18.046999 2020,317,18:14:18.000000 -0.047 47.999 9N_C7035GPZ 2020,317,23:59:59.999998 2020,319,00:00:00.000000 1.0d 86399999 9N_C7035__GPZ 2020,320,00:00:00.046998 2020,320,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,321,00:00:00.046998 2020,321,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,322,00:00:00.046998 2020,322,00:00:00.000000 -0.047 47.998 9N_C7035__GPZ 2020,323,00:00:00.046998 2020,323,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,324,00:00:00.046998 2020,324,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,324,23:59:59.999998 2020,326,00:00:00.000000 1.0d 86399999 9N_C7035__GPZ 2020,327,00:00:00.046998 2020,327,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,328,00:00:00.046998 2020,328,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,329,00:00:00.046998 2020,329,00:00:00.000000 -0.047 47.998 9N_C7035__GPZ 2020,330,00:00:00.046998 2020,330,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,331,00:00:00.046998 2020,331,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,331,23:59:59.999998 2020,333,00:00:00.000000 1.0d 86399999 9N_C7035__GPZ 2020,334,00:00:00.046998 2020,334,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,335,00:00:00.046998 2020,335,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,336,00:00:00.046998 2020,336,00:00:00.000000 -0.047 47.998 9N_C7035__GPZ 2020,337,00:00:00.046998 2020,337,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,338,00:00:00.046998 2020,338,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,339,00:00:00.046998 2020,339,00:00:00.000000 -0.047 47.998 9N_C7035__GPZ 2020,340,00:00:00.046998 2020,340,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,340,23:59:59.999998 2020,342,00:00:00.000000 1.0d 86399999 Total: 69 gap(s)

    • Duplicated on webservices: http://service.iris.edu/ph5ws/dataselect/1/query?reqtype=FDSN&format=mseed&net=9N&sta=C7035&cha=*&starttime=2020-01-01&endtime=2020-12-31&nodata=404

    • msi -G ph5ws-dataselect_2022-02-07t17_35_41z.mseed Source Last Sample Next Sample Gap Samples 9N_C7035GP1 2020,316,18:42:05.002000 2020,316,18:14:18.000000 -1667 1667003 9N_C7035__GP1 2020,317,18:14:18.046999 2020,317,18:14:18.000000 -0.047 47.999 9N_C7035GP1 2020,317,23:59:59.999998 2020,319,00:00:00.000000 1.0d 86399999 9N_C7035GP1 2020,320,00:00:00.046998 2020,320,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,321,00:00:00.046998 2020,321,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,322,00:00:00.046998 2020,322,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,323,00:00:00.046998 2020,323,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,324,00:00:00.046998 2020,324,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,324,23:59:59.999998 2020,326,00:00:00.000000 1.0d 86399999 9N_C7035GP1 2020,327,00:00:00.046998 2020,327,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,328,00:00:00.046998 2020,328,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,329,00:00:00.046998 2020,329,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,330,00:00:00.046998 2020,330,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,331,00:00:00.046998 2020,331,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,331,23:59:59.999998 2020,333,00:00:00.000000 1.0d 86399999 9N_C7035GP1 2020,334,00:00:00.046998 2020,334,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,335,00:00:00.046998 2020,335,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,336,00:00:00.046998 2020,336,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,337,00:00:00.046998 2020,337,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,338,00:00:00.046998 2020,338,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,339,00:00:00.046998 2020,339,00:00:00.000000 -0.047 47.998 9N_C7035GP1 2020,340,00:00:00.046998 2020,340,00:00:00.000000 -0.047 47.998 9N_C7035__GP1 2020,340,23:59:59.999998 2020,342,00:00:00.000000 1.0d 86399999 9N_C7035GP2 2020,316,18:42:05.002000 2020,316,18:14:18.000000 -1667 1667003 9N_C7035GP2 2020,317,18:14:18.046999 2020,317,18:14:18.000000 -0.047 47.999 9N_C7035__GP2 2020,317,23:59:59.999998 2020,319,00:00:00.000000 1.0d 86399999 9N_C7035GP2 2020,320,00:00:00.046998 2020,320,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,321,00:00:00.046998 2020,321,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,322,00:00:00.046998 2020,322,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,323,00:00:00.046998 2020,323,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,324,00:00:00.046998 2020,324,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,324,23:59:59.999998 2020,326,00:00:00.000000 1.0d 86399999 9N_C7035GP2 2020,327,00:00:00.046998 2020,327,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,328,00:00:00.046998 2020,328,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,329,00:00:00.046998 2020,329,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,330,00:00:00.046998 2020,330,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,331,00:00:00.046998 2020,331,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,331,23:59:59.999998 2020,333,00:00:00.000000 1.0d 86399999 9N_C7035GP2 2020,334,00:00:00.046998 2020,334,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,335,00:00:00.046998 2020,335,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,336,00:00:00.046998 2020,336,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,337,00:00:00.046998 2020,337,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,338,00:00:00.046998 2020,338,00:00:00.000000 -0.047 47.998 9N_C7035__GP2 2020,339,00:00:00.046998 2020,339,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,340,00:00:00.046998 2020,340,00:00:00.000000 -0.047 47.998 9N_C7035GP2 2020,340,23:59:59.999998 2020,342,00:00:00.000000 1.0d 86399999 9N_C7035__GPZ 2020,316,18:42:05.002000 2020,316,18:14:18.000000 -1667 1667003 9N_C7035GPZ 2020,317,18:14:18.046999 2020,317,18:14:18.000000 -0.047 47.999 9N_C7035GPZ 2020,317,23:59:59.999998 2020,319,00:00:00.000000 1.0d 86399999 9N_C7035__GPZ 2020,320,00:00:00.046998 2020,320,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,321,00:00:00.046998 2020,321,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,322,00:00:00.046998 2020,322,00:00:00.000000 -0.047 47.998 9N_C7035__GPZ 2020,323,00:00:00.046998 2020,323,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,324,00:00:00.046998 2020,324,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,324,23:59:59.999998 2020,326,00:00:00.000000 1.0d 86399999 9N_C7035__GPZ 2020,327,00:00:00.046998 2020,327,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,328,00:00:00.046998 2020,328,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,329,00:00:00.046998 2020,329,00:00:00.000000 -0.047 47.998 9N_C7035__GPZ 2020,330,00:00:00.046998 2020,330,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,331,00:00:00.046998 2020,331,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,331,23:59:59.999998 2020,333,00:00:00.000000 1.0d 86399999 9N_C7035__GPZ 2020,334,00:00:00.046998 2020,334,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,335,00:00:00.046998 2020,335,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,336,00:00:00.046998 2020,336,00:00:00.000000 -0.047 47.998 9N_C7035__GPZ 2020,337,00:00:00.046998 2020,337,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,338,00:00:00.046998 2020,338,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,339,00:00:00.046998 2020,339,00:00:00.000000 -0.047 47.998 9N_C7035__GPZ 2020,340,00:00:00.046998 2020,340,00:00:00.000000 -0.047 47.998 9N_C7035GPZ 2020,340,23:59:59.999998 2020,342,00:00:00.000000 1.0d 86399999 Total: 69 gap(s)

  2. I know the data should not be missing entire days, so can one of these missing days be extracted? This example is day 2020.318/2020-11-13.

    • PIC server: ph5toms -n master.ph5 -a 002 --station C7035 -s 2020-11-13T00:00:00 -t 2020-11-13T23:59:59 -o mseed_stnC7035_missingday_test1 -F MSEED

    • msi -S 9N* DCC|2022,039 9N|C7035||GP1|2020,318,00:00:00.000998|2020,318,18:14:18.046999||1000|65658047|||||||2022,039 9N|C7035||GP1|2020,318,18:14:18.000000|2020,318,23:59:58.999998||1000|20741001|||||||2022,039 9N|C7035||GP2|2020,318,00:00:00.000998|2020,318,18:14:18.046999||1000|65658047|||||||2022,039 9N|C7035||GP2|2020,318,18:14:18.000000|2020,318,23:59:58.999998||1000|20741001|||||||2022,039 9N|C7035||GPZ|2020,318,00:00:00.000998|2020,318,18:14:18.046999||1000|65658047|||||||2022,039 9N|C7035||GPZ|2020,318,18:14:18.000000|2020,318,23:59:58.999998||1000|20741001|||||||2022,039

    • msi -G 9N* Source Last Sample Next Sample Gap Samples 9N_C7035GP1 2020,318,18:14:18.046999 2020,318,18:14:18.000000 -0.047 47.999 9N_C7035__GP2 2020,318,18:14:18.046999 2020,318,18:14:18.000000 -0.047 47.999 9N_C7035GPZ 2020,318,18:14:18.046999 2020,318,18:14:18.000000 -0.047 47.999 Total: 3 gap(s)

    • Duplicated on webservices: http://service.iris.edu/ph5ws/dataselect/1/query?reqtype=FDSN&format=mseed&net=9N&sta=C7035&cha=*&starttime=2020-11-13T00:00:00&endtime=2020-11-13T23:59:59&nodata=404

    • msi -S ph5ws-dataselect_2022-02-08t14_31_19z.mseed DCC|2022,039 9N|C7035||GP1|2020,318,00:00:00.000998|2020,318,18:14:18.046999||1000|65658047|||||||2022,039 9N|C7035||GP1|2020,318,18:14:18.000000|2020,318,23:59:58.999998||1000|20741001|||||||2022,039 9N|C7035||GP2|2020,318,00:00:00.000998|2020,318,18:14:18.046999||1000|65658047|||||||2022,039 9N|C7035||GP2|2020,318,18:14:18.000000|2020,318,23:59:58.999998||1000|20741001|||||||2022,039 9N|C7035||GPZ|2020,318,00:00:00.000998|2020,318,18:14:18.046999||1000|65658047|||||||2022,039 9N|C7035||GPZ|2020,318,18:14:18.000000|2020,318,23:59:58.999998||1000|20741001|||||||2022,039

    • msi -G ph5ws-dataselect_2022-02-08t14_31_19z.mseed Source Last Sample Next Sample Gap Samples 9N_C7035GP1 2020,318,18:14:18.046999 2020,318,18:14:18.000000 -0.047 47.999 9N_C7035__GP2 2020,318,18:14:18.046999 2020,318,18:14:18.000000 -0.047 47.999 9N_C7035GPZ 2020,318,18:14:18.046999 2020,318,18:14:18.000000 -0.047 47.999 Total: 3 gap(s)

The answer appears to be yes.

  1. Can the query go outside the originally missing day (2020-11-13) at all and still extract data for that day?

    • PIC server: ph5toms -n master.ph5 -a 002 --station C7035 -s 2020-11-12T23:45:00 -t 2020-11-14T00:15:00 -o mseed_stnC7035_missingday_test2span -F MSEED

    • msi -S 9N* DCC|2022,039 9N|C7035||GP1|2020,317,23:45:00.000998|2020,317,23:59:59.999998||1000|900000|||||||2022,039 9N|C7035||GP1|2020,318,23:44:18.010998|2020,318,23:58:36.020998||1000|858011|||||||2022,039 9N|C7035||GP1|2020,319,00:00:00.000000|2020,319,00:14:59.999000||1000|900000|||||||2022,039 9N|C7035||GP2|2020,317,23:45:00.000998|2020,317,23:59:59.999998||1000|900000|||||||2022,039 9N|C7035||GP2|2020,318,23:44:18.010998|2020,318,23:58:36.020998||1000|858011|||||||2022,039 9N|C7035||GP2|2020,319,00:00:00.000000|2020,319,00:14:59.999000||1000|900000|||||||2022,039 9N|C7035||GPZ|2020,317,23:45:00.000998|2020,317,23:59:59.999998||1000|900000|||||||2022,039 9N|C7035||GPZ|2020,318,23:44:18.010998|2020,318,23:58:36.020998||1000|858011|||||||2022,039 9N|C7035||GPZ|2020,319,00:00:00.000000|2020,319,00:14:59.999000||1000|900000|||||||2022,039

    • msi -G 9N* Source Last Sample Next Sample Gap Samples 9N_C7035GP1 2020,317,23:59:59.999998 2020,318,23:44:18.010998 23.7h 85458010 9N_C7035__GP1 2020,318,23:58:36.020998 2020,319,00:00:00.000000 83.98 83978.002 9N_C7035GP2 2020,317,23:59:59.999998 2020,318,23:44:18.010998 23.7h 85458010 9N_C7035GP2 2020,318,23:58:36.020998 2020,319,00:00:00.000000 83.98 83978.002 9N_C7035__GPZ 2020,317,23:59:59.999998 2020,318,23:44:18.010998 23.7h 85458010 9N_C7035GPZ 2020,318,23:58:36.020998 2020,319,00:00:00.000000 83.98 83978.002 Total: 6 gap(s)

    • Duplicated on webservices: http://service.iris.edu/ph5ws/dataselect/1/query?reqtype=FDSN&format=mseed&net=9N&sta=C7035&cha=*&starttime=2020-11-12T23:45:00&endtime=2020-11-14T00:15:00&nodata=404

    • msi -S ph5ws-dataselect_2022-02-08t15_30_55z.mseed DCC|2022,039 9N|C7035||GP1|2020,317,23:45:00.000998|2020,317,23:59:59.999998||1000|900000|||||||2022,039 9N|C7035||GP1|2020,318,23:44:18.010998|2020,318,23:58:36.020998||1000|858011|||||||2022,039 9N|C7035||GP1|2020,319,00:00:00.000000|2020,319,00:14:59.999000||1000|900000|||||||2022,039 9N|C7035||GP2|2020,317,23:45:00.000998|2020,317,23:59:59.999998||1000|900000|||||||2022,039 9N|C7035||GP2|2020,318,23:44:18.010998|2020,318,23:58:36.020998||1000|858011|||||||2022,039 9N|C7035||GP2|2020,319,00:00:00.000000|2020,319,00:14:59.999000||1000|900000|||||||2022,039 9N|C7035||GPZ|2020,317,23:45:00.000998|2020,317,23:59:59.999998||1000|900000|||||||2022,039 9N|C7035||GPZ|2020,318,23:44:18.010998|2020,318,23:58:36.020998||1000|858011|||||||2022,039 9N|C7035||GPZ|2020,319,00:00:00.000000|2020,319,00:14:59.999000||1000|900000|||||||2022,039

    • msi -G ph5ws-dataselect_2022-02-08t15_30_55z.mseed Source Last Sample Next Sample Gap Samples 9N_C7035GP1 2020,317,23:59:59.999998 2020,318,23:44:18.010998 23.7h 85458010 9N_C7035__GP1 2020,318,23:58:36.020998 2020,319,00:00:00.000000 83.98 83978.002 9N_C7035GP2 2020,317,23:59:59.999998 2020,318,23:44:18.010998 23.7h 85458010 9N_C7035GP2 2020,318,23:58:36.020998 2020,319,00:00:00.000000 83.98 83978.002 9N_C7035__GPZ 2020,317,23:59:59.999998 2020,318,23:44:18.010998 23.7h 85458010 9N_C7035GPZ 2020,318,23:58:36.020998 2020,319,00:00:00.000000 83.98 83978.002 Total: 6 gap(s)

It appears mostly not, although it is interesting that the gap is not exactly 1 day on this attempt.

Experiment 21-014

Station 7051: only PIC server queries are used, and for the sake of space only one channel is queried on msi.

  1. Entire deployment

    • ph5toms -n master.ph5 -a 007 --station 7051 -o mseed_stn7051_test2 -F MSEED
    • msi -G ZV*GP1* Source Last Sample Next Sample Gap Samples ZV_7051GP1 2021,139,20:47:36.000000 2021,139,20:47:36.000000 -0 1 ZV_7051__GP1 2021,139,21:17:36.000000 2021,139,21:17:36.000000 -0 1 ZV_7051GP1 2021,139,21:47:36.000000 2021,139,21:47:36.000000 -0 1 ZV_7051GP1 2021,139,22:17:36.000000 2021,139,22:17:36.000000 -0 1 ZV_7051__GP1 2021,139,22:47:36.000000 2021,139,22:47:36.000000 -0 1 ZV_7051GP1 2021,139,23:17:36.000000 2021,139,23:17:36.000000 -0 1 ZV_7051GP1 2021,139,23:47:36.000000 2021,139,23:47:36.000000 -0 1 ZV_7051__GP1 2021,139,23:59:59.999500 2021,141,00:00:00.000000 1.0d 1.728e+08 ZV_7051GP1 2021,141,00:30:00.000000 2021,141,00:30:00.000000 -0 1 ZV_7051GP1 2021,141,01:00:00.000000 2021,141,01:00:00.000000 -0 1 ZV_7051__GP1 2021,141,01:30:00.000000 2021,141,01:30:00.000000 -0 1 ZV_7051GP1 2021,141,02:00:00.000000 2021,141,02:00:00.000000 -0 1 ZV_7051GP1 2021,141,02:30:00.000000 2021,141,02:30:00.000000 -0 1 ZV_7051__GP1 2021,141,03:00:00.000000 2021,141,03:00:00.000000 -0 1 ZV_7051GP1 2021,141,03:30:00.000000 2021,141,03:30:00.000000 -0 1 ZV_7051GP1 2021,141,04:00:00.000000 2021,141,04:00:00.000000 -0 1 ZV_7051__GP1 2021,141,04:30:00.000000 2021,141,04:30:00.000000 -0 1 ZV_7051GP1 2021,141,05:00:00.000000 2021,141,05:00:00.000000 -0 1 ZV_7051GP1 2021,141,05:30:00.000000 2021,141,05:30:00.000000 -0 1 ZV_7051__GP1 2021,141,06:00:00.000000 2021,141,06:00:00.000000 -0 1 ZV_7051GP1 2021,141,06:30:00.000000 2021,141,06:30:00.000000 -0 1 ZV_7051GP1 2021,141,07:00:00.000000 2021,141,07:00:00.000000 -0 1 ZV_7051__GP1 2021,141,07:30:00.000000 2021,141,07:30:00.000000 -0 1 ZV_7051GP1 2021,141,08:00:00.000000 2021,141,08:00:00.000000 -0 1 ZV_7051GP1 2021,141,08:30:00.000000 2021,141,08:30:00.000000 -0 1 ZV_7051__GP1 2021,141,09:00:00.000000 2021,141,09:00:00.000000 -0 1 ZV_7051GP1 2021,141,09:30:00.000000 2021,141,09:30:00.000000 -0 1 ZV_7051GP1 2021,141,10:00:00.000000 2021,141,10:00:00.000000 -0 1 ZV_7051__GP1 2021,141,10:30:00.000000 2021,141,10:30:00.000000 -0 1 ZV_7051GP1 2021,141,11:00:00.000000 2021,141,11:00:00.000000 -0 1 ZV_7051GP1 2021,141,11:30:00.000000 2021,141,11:30:00.000000 -0 1 ZV_7051__GP1 2021,141,12:00:00.000000 2021,141,12:00:00.000000 -0 1 ZV_7051GP1 2021,141,12:30:00.000000 2021,141,12:30:00.000000 -0 1 ZV_7051GP1 2021,141,13:00:00.000000 2021,141,13:00:00.000000 -0 1 ZV_7051__GP1 2021,141,13:30:00.000000 2021,141,13:30:00.000000 -0 1 ZV_7051GP1 2021,141,14:00:00.000000 2021,141,14:00:00.000000 -0 1 ZV_7051GP1 2021,141,14:30:00.000000 2021,141,14:30:00.000000 -0 1 ZV_7051__GP1 2021,141,15:00:00.000000 2021,141,15:00:00.000000 -0 1 ZV_7051GP1 2021,141,15:30:00.000000 2021,141,15:30:00.000000 -0 1 ZV_7051GP1 2021,141,16:00:00.000000 2021,141,16:00:00.000000 -0 1 ZV_7051__GP1 2021,141,16:30:00.000000 2021,141,16:30:00.000000 -0 1 ZV_7051GP1 2021,141,17:00:00.000000 2021,141,17:00:00.000000 -0 1 ZV_7051GP1 2021,141,17:30:00.000000 2021,141,17:30:00.000000 -0 1 ZV_7051__GP1 2021,141,18:00:00.000000 2021,141,18:00:00.000000 -0 1 ZV_7051GP1 2021,141,18:30:00.000000 2021,141,18:30:00.000000 -0 1 ZV_7051GP1 2021,141,19:00:00.000000 2021,141,19:00:00.000000 -0 1 ZV_7051__GP1 2021,141,19:30:00.000000 2021,141,19:30:00.000000 -0 1 ZV_7051GP1 2021,141,20:00:00.000000 2021,141,20:00:00.000000 -0 1 ZV_7051GP1 2021,141,20:30:00.000000 2021,141,20:30:00.000000 -0 1 ZV_7051__GP1 2021,141,21:00:00.000000 2021,141,21:00:00.000000 -0 1 ZV_7051GP1 2021,141,21:30:00.000000 2021,141,21:30:00.000000 -0 1 ZV_7051GP1 2021,141,22:00:00.000000 2021,141,22:00:00.000000 -0 1 ZV_7051__GP1 2021,141,22:30:00.000000 2021,141,22:30:00.000000 -0 1 ZV_7051GP1 2021,141,23:00:00.000000 2021,141,23:00:00.000000 -0 1 ZV_7051GP1 2021,141,23:30:00.000000 2021,141,23:30:00.000000 -0 1 ZV_7051__GP1 2021,142,00:00:00.000000 2021,142,00:00:00.000000 -0 1 ZV_7051GP1 2021,142,00:30:00.000000 2021,142,00:30:00.000000 -0 1 ZV_7051GP1 2021,142,01:00:00.000000 2021,142,01:00:00.000000 -0 1 ZV_7051__GP1 2021,142,01:30:00.000000 2021,142,01:30:00.000000 -0 1 ZV_7051GP1 2021,142,02:00:00.000000 2021,142,02:00:00.000000 -0 1 ZV_7051GP1 2021,142,02:30:00.000000 2021,142,02:30:00.000000 -0 1 ZV_7051__GP1 2021,142,03:00:00.000000 2021,142,03:00:00.000000 -0 1 ZV_7051GP1 2021,142,03:30:00.000000 2021,142,03:30:00.000000 -0 1 ZV_7051GP1 2021,142,04:00:00.000000 2021,142,04:00:00.000000 -0 1 ZV_7051__GP1 2021,142,04:30:00.000000 2021,142,04:30:00.000000 -0 1 ZV_7051GP1 2021,142,05:00:00.000000 2021,142,05:00:00.000000 -0 1 ZV_7051GP1 2021,142,05:30:00.000000 2021,142,05:30:00.000000 -0 1 ZV_7051__GP1 2021,142,06:00:00.000000 2021,142,06:00:00.000000 -0 1 ZV_7051GP1 2021,142,06:30:00.000000 2021,142,06:30:00.000000 -0 1 ZV_7051GP1 2021,142,07:00:00.000000 2021,142,07:00:00.000000 -0 1 ZV_7051__GP1 2021,142,07:30:00.000000 2021,142,07:30:00.000000 -0 1 ZV_7051GP1 2021,142,08:00:00.000000 2021,142,08:00:00.000000 -0 1 ZV_7051GP1 2021,142,08:30:00.000000 2021,142,08:30:00.000000 -0 1 ZV_7051__GP1 2021,142,09:00:00.000000 2021,142,09:00:00.000000 -0 1 ZV_7051GP1 2021,142,09:30:00.000000 2021,142,09:30:00.000000 -0 1 ZV_7051GP1 2021,142,10:00:00.000000 2021,142,10:00:00.000000 -0 1 ZV_7051__GP1 2021,142,10:30:00.000000 2021,142,10:30:00.000000 -0 1 ZV_7051GP1 2021,142,11:00:00.000000 2021,142,11:00:00.000000 -0 1 ZV_7051GP1 2021,142,11:30:00.000000 2021,142,11:30:00.000000 -0 1 ZV_7051__GP1 2021,142,12:00:00.000000 2021,142,12:00:00.000000 -0 1 ZV_7051GP1 2021,142,12:30:00.000000 2021,142,12:30:00.000000 -0 1 ZV_7051GP1 2021,142,13:00:00.000000 2021,142,13:00:00.000000 -0 1 ZV_7051__GP1 2021,142,13:30:00.000000 2021,142,13:30:00.000000 -0 1 ZV_7051GP1 2021,142,14:00:00.000000 2021,142,14:00:00.000000 -0 1 ZV_7051__GP1 2021,142,14:30:00.000000 2021,142,14:30:00.000000 -0 1 Total: 85 gap(s)

  2. There should not be an entire missing day here either, so can the missing day be extracted by specifying only the missing time?

    • ph5toms -n master.ph5 -a 007 --station 7051 -s 2021-05-20T00:00:00 -t 2021-05-20T23:59:59 -o mseed_stn7051_missingday1b -F MSEED mseed_stn7051_missingday1b$ ls ZV.7051..GP1.2021-05-20T00.00.00.ms ZV.7051..GPZ.2021-05-20T00.00.00.ms ZV.7051..GP2.2021-05-20T00.00.00.ms
    • msi -S ZV*GP1* DCC|2022,041 ZV|7051||GP1|2021,140,00:00:00.000000|2021,140,00:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,00:00:00.000000|2021,140,00:17:36.000000||2000|2112001|||||||2022,041 ZV|7051||GP1|2021,140,00:30:00.000000|2021,140,01:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,01:00:00.000000|2021,140,01:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,01:30:00.000000|2021,140,02:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,02:00:00.000000|2021,140,02:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,02:30:00.000000|2021,140,03:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,03:00:00.000000|2021,140,03:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,03:30:00.000000|2021,140,04:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,04:00:00.000000|2021,140,04:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,04:30:00.000000|2021,140,05:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,05:00:00.000000|2021,140,05:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,05:30:00.000000|2021,140,06:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,06:00:00.000000|2021,140,06:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,06:30:00.000000|2021,140,07:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,07:00:00.000000|2021,140,07:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,07:30:00.000000|2021,140,08:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,08:00:00.000000|2021,140,08:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,08:30:00.000000|2021,140,09:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,09:00:00.000000|2021,140,09:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,09:30:00.000000|2021,140,10:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,10:00:00.000000|2021,140,10:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,10:30:00.000000|2021,140,11:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,11:00:00.000000|2021,140,11:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,11:30:00.000000|2021,140,12:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,12:00:00.000000|2021,140,12:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,12:30:00.000000|2021,140,13:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,13:00:00.000000|2021,140,13:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,13:30:00.000000|2021,140,14:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,14:00:00.000000|2021,140,14:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,14:30:00.000000|2021,140,15:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,15:00:00.000000|2021,140,15:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,15:30:00.000000|2021,140,16:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,16:00:00.000000|2021,140,16:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,16:30:00.000000|2021,140,17:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,17:00:00.000000|2021,140,17:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,17:30:00.000000|2021,140,18:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,18:00:00.000000|2021,140,18:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,18:30:00.000000|2021,140,19:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,19:00:00.000000|2021,140,19:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,19:30:00.000000|2021,140,20:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,20:00:00.000000|2021,140,20:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,20:30:00.000000|2021,140,21:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,21:00:00.000000|2021,140,21:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,21:30:00.000000|2021,140,22:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,22:00:00.000000|2021,140,22:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,22:30:00.000000|2021,140,23:00:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,23:00:00.000000|2021,140,23:30:00.000000||2000|3600001|||||||2022,041 ZV|7051||GP1|2021,140,23:30:00.000000|2021,140,23:59:58.999500||2000|3598000|||||||2022,041
    • msi -G ZV*GP1* Source Last Sample Next Sample Gap Samples ZV_7051GP1 2021,140,00:30:00.000000 2021,140,00:00:00.000000 -1056 2112002 ZV_7051__GP1 2021,140,00:17:36.000000 2021,140,00:30:00.000000 744 1487999 ZV_7051GP1 2021,140,01:00:00.000000 2021,140,01:00:00.000000 -0 1 ZV_7051GP1 2021,140,01:30:00.000000 2021,140,01:30:00.000000 -0 1 ZV_7051__GP1 2021,140,02:00:00.000000 2021,140,02:00:00.000000 -0 1 ZV_7051GP1 2021,140,02:30:00.000000 2021,140,02:30:00.000000 -0 1 ZV_7051GP1 2021,140,03:00:00.000000 2021,140,03:00:00.000000 -0 1 ZV_7051__GP1 2021,140,03:30:00.000000 2021,140,03:30:00.000000 -0 1 ZV_7051GP1 2021,140,04:00:00.000000 2021,140,04:00:00.000000 -0 1 ZV_7051GP1 2021,140,04:30:00.000000 2021,140,04:30:00.000000 -0 1 ZV_7051__GP1 2021,140,05:00:00.000000 2021,140,05:00:00.000000 -0 1 ZV_7051GP1 2021,140,05:30:00.000000 2021,140,05:30:00.000000 -0 1 ZV_7051GP1 2021,140,06:00:00.000000 2021,140,06:00:00.000000 -0 1 ZV_7051__GP1 2021,140,06:30:00.000000 2021,140,06:30:00.000000 -0 1 ZV_7051GP1 2021,140,07:00:00.000000 2021,140,07:00:00.000000 -0 1 ZV_7051GP1 2021,140,07:30:00.000000 2021,140,07:30:00.000000 -0 1 ZV_7051__GP1 2021,140,08:00:00.000000 2021,140,08:00:00.000000 -0 1 ZV_7051GP1 2021,140,08:30:00.000000 2021,140,08:30:00.000000 -0 1 ZV_7051GP1 2021,140,09:00:00.000000 2021,140,09:00:00.000000 -0 1 ZV_7051__GP1 2021,140,09:30:00.000000 2021,140,09:30:00.000000 -0 1 ZV_7051GP1 2021,140,10:00:00.000000 2021,140,10:00:00.000000 -0 1 ZV_7051GP1 2021,140,10:30:00.000000 2021,140,10:30:00.000000 -0 1 ZV_7051__GP1 2021,140,11:00:00.000000 2021,140,11:00:00.000000 -0 1 ZV_7051GP1 2021,140,11:30:00.000000 2021,140,11:30:00.000000 -0 1 ZV_7051GP1 2021,140,12:00:00.000000 2021,140,12:00:00.000000 -0 1 ZV_7051__GP1 2021,140,12:30:00.000000 2021,140,12:30:00.000000 -0 1 ZV_7051GP1 2021,140,13:00:00.000000 2021,140,13:00:00.000000 -0 1 ZV_7051GP1 2021,140,13:30:00.000000 2021,140,13:30:00.000000 -0 1 ZV_7051__GP1 2021,140,14:00:00.000000 2021,140,14:00:00.000000 -0 1 ZV_7051GP1 2021,140,14:30:00.000000 2021,140,14:30:00.000000 -0 1 ZV_7051GP1 2021,140,15:00:00.000000 2021,140,15:00:00.000000 -0 1 ZV_7051__GP1 2021,140,15:30:00.000000 2021,140,15:30:00.000000 -0 1 ZV_7051GP1 2021,140,16:00:00.000000 2021,140,16:00:00.000000 -0 1 ZV_7051GP1 2021,140,16:30:00.000000 2021,140,16:30:00.000000 -0 1 ZV_7051__GP1 2021,140,17:00:00.000000 2021,140,17:00:00.000000 -0 1 ZV_7051GP1 2021,140,17:30:00.000000 2021,140,17:30:00.000000 -0 1 ZV_7051GP1 2021,140,18:00:00.000000 2021,140,18:00:00.000000 -0 1 ZV_7051__GP1 2021,140,18:30:00.000000 2021,140,18:30:00.000000 -0 1 ZV_7051GP1 2021,140,19:00:00.000000 2021,140,19:00:00.000000 -0 1 ZV_7051GP1 2021,140,19:30:00.000000 2021,140,19:30:00.000000 -0 1 ZV_7051__GP1 2021,140,20:00:00.000000 2021,140,20:00:00.000000 -0 1 ZV_7051GP1 2021,140,20:30:00.000000 2021,140,20:30:00.000000 -0 1 ZV_7051GP1 2021,140,21:00:00.000000 2021,140,21:00:00.000000 -0 1 ZV_7051__GP1 2021,140,21:30:00.000000 2021,140,21:30:00.000000 -0 1 ZV_7051GP1 2021,140,22:00:00.000000 2021,140,22:00:00.000000 -0 1 ZV_7051GP1 2021,140,22:30:00.000000 2021,140,22:30:00.000000 -0 1 ZV_7051__GP1 2021,140,23:00:00.000000 2021,140,23:00:00.000000 -0 1 ZV_7051GP1 2021,140,23:30:00.000000 2021,140,23:30:00.000000 -0 1 Total: 48 gap(s)

Again, the answer appears to be yes or mostly yes.

  1. Can the query go beyond these bounds and still extract data from 2021-05-20?
    • ph5toms -n master.ph5 -a 007 --station 7051 -s 2021-05-19T23:45:00 -t 2021-05-21T00:05:00 -o mseed_stn7051_test5 -F MSEED
    • mseed_stn7051_test5$ msi -S ZV*GP1* DCC|2022,041 ZV|7051||GP1|2021,139,23:45:00.000000|2021,139,23:47:36.000000||2000|312001|||||||2022,041 ZV|7051||GP1|2021,139,23:47:36.000000|2021,139,23:59:59.999500||2000|1488000|||||||2022,041 ZV|7051||GP1|2021,140,23:30:00.000000|2021,140,23:30:00.000000||2000|1|||||||2022,041 ZV|7051||GP1|2021,141,00:00:00.000000|2021,141,00:04:59.999500||2000|600000|||||||2022,041
    • msi -G ZV*GP1* Source Last Sample Next Sample Gap Samples ZV_7051GP1 2021,139,23:47:36.000000 2021,139,23:47:36.000000 -0 1 ZV_7051__GP1 2021,139,23:59:59.999500 2021,140,23:30:00.000000 23.5h 1.692e+08 ZV_7051GP1 2021,140,23:30:00.000000 2021,141,00:00:00.000000 1800 3599999 Total: 3 gap(s)

Again, the answer appears to be mostly not.

Expected behavior All data that is contained in the PH5 and is within the metadata limits and/or the ph5toms query limits (whichever is more restrictive) should be extracted for the user.

Additional context It's possible this is related to one or more already-created issue(s). I would also like to hear ideas for the most effective way(s) to proceed with determining how widespread this bug is (ph5toms for full deployment of X% of an experiment's stations? but what is X? SmartSolo only?), if that is part of the strategy for creating a fix. I am happy to do keep doing more queries from the PIC server side.

hrotman-pic commented 2 years ago

Example of unexpected gap from a Fairfield experiment with several input files: msi -G DPZ Source Last Sample Next Sample Gap Samples Z4_2021DPZ 2021,155,11:54:15.077998 2021,155,12:22:57.809998 1723 861365 Z4_2021__DPZ 2021,156,11:22:57.807998 2021,156,11:00:49.861999 -1328 663974 Z4_2021DPZ 2021,156,12:30:49.859999 2021,156,12:08:41.874000 -1328 663994 Z4_2021DPZ 2021,156,17:38:41.872000 2021,156,17:35:59.937999 -161.9 80968 Z4_2021__DPZ 2021,158,09:05:59.935999 2021,158,08:43:59.885998 -1320 660026 Z4_2021DPZ 2021,158,11:43:59.883998 2021,158,11:22:07.871999 -1312 656007 Z4_2021DPZ 2021,161,23:59:59.999999 2021,162,11:00:48.885998 11.0h 19824442 Z4_2021__DPZ 2021,163,00:30:48.883998 2021,163,01:42:27.767998 1.2h 2149441 Z4_2021DPZ 2021,163,23:59:59.999998 2021,164,16:57:53.875998 17.0h 30536937 Z4_2021__DPZ 2021,165,10:27:53.873998 2021,165,10:09:33.880000 -1100 549998 Z4_2021__DPZ 2021,166,11:09:33.878000 2021,166,10:57:49.895998 -704 351992 Total: 11 gap(s)

These are not full-day gaps like SmartSolo, but that is likely related to the length of the respective raw files.

timronan commented 2 years ago

I ran ph5validate on experiment 21-014 and received a number of errors stating that the DAS tables were out of order.

As suggested by the output message I ran: fix_das_t_order, which executed with no issues.

After executing fix_das_t_order ph5_validate was reran and the there were no errors were reported: (this check shows that PH5 validate works as expected).

After running ph5_validate and receiving no errors, the test below was rerun:

ph5toms -n master.ph5 -a 007 --station 7051 -o mseed_stn7051_test2 -F MSEED
msi -G ZV*GP1* | grep ,140,

Resulted in:

ZV_7051__GP1      2021,140,00:17:36.000000 2021,140,00:00:00.000000 -1056 2112001
ZV_7051__GP1      2021,140,00:30:00.000000 2021,140,00:30:00.000000 -0   1
ZV_7051__GP1      2021,140,01:00:00.000000 2021,140,01:00:00.000000 -0   1
ZV_7051__GP1      2021,140,01:30:00.000000 2021,140,01:30:00.000000 -0   1
ZV_7051__GP1      2021,140,02:00:00.000000 2021,140,02:00:00.000000 -0   1
ZV_7051__GP1      2021,140,02:30:00.000000 2021,140,02:30:00.000000 -0   1
ZV_7051__GP1      2021,140,03:00:00.000000 2021,140,03:00:00.000000 -0   1
ZV_7051__GP1      2021,140,03:30:00.000000 2021,140,03:30:00.000000 -0   1
ZV_7051__GP1      2021,140,04:00:00.000000 2021,140,04:00:00.000000 -0   1
ZV_7051__GP1      2021,140,04:30:00.000000 2021,140,04:30:00.000000 -0   1
ZV_7051__GP1      2021,140,05:00:00.000000 2021,140,05:00:00.000000 -0   1
ZV_7051__GP1      2021,140,05:30:00.000000 2021,140,05:30:00.000000 -0   1
ZV_7051__GP1      2021,140,06:00:00.000000 2021,140,06:00:00.000000 -0   1
ZV_7051__GP1      2021,140,06:30:00.000000 2021,140,06:30:00.000000 -0   1
ZV_7051__GP1      2021,140,07:00:00.000000 2021,140,07:00:00.000000 -0   1
ZV_7051__GP1      2021,140,07:30:00.000000 2021,140,07:30:00.000000 -0   1
ZV_7051__GP1      2021,140,08:00:00.000000 2021,140,08:00:00.000000 -0   1
ZV_7051__GP1      2021,140,08:30:00.000000 2021,140,08:30:00.000000 -0   1
ZV_7051__GP1      2021,140,09:00:00.000000 2021,140,09:00:00.000000 -0   1
ZV_7051__GP1      2021,140,09:30:00.000000 2021,140,09:30:00.000000 -0   1
ZV_7051__GP1      2021,140,10:00:00.000000 2021,140,10:00:00.000000 -0   1
ZV_7051__GP1      2021,140,10:30:00.000000 2021,140,10:30:00.000000 -0   1
ZV_7051__GP1      2021,140,11:00:00.000000 2021,140,11:00:00.000000 -0   1
ZV_7051__GP1      2021,140,11:30:00.000000 2021,140,11:30:00.000000 -0   1
ZV_7051__GP1      2021,140,12:00:00.000000 2021,140,12:00:00.000000 -0   1
ZV_7051__GP1      2021,140,12:30:00.000000 2021,140,12:30:00.000000 -0   1
ZV_7051__GP1      2021,140,13:00:00.000000 2021,140,13:00:00.000000 -0   1
ZV_7051__GP1      2021,140,13:30:00.000000 2021,140,13:30:00.000000 -0   1
ZV_7051__GP1      2021,140,14:00:00.000000 2021,140,14:00:00.000000 -0   1
ZV_7051__GP1      2021,140,14:30:00.000000 2021,140,14:30:00.000000 -0   1
ZV_7051__GP1      2021,140,15:00:00.000000 2021,140,15:00:00.000000 -0   1
ZV_7051__GP1      2021,140,15:30:00.000000 2021,140,15:30:00.000000 -0   1
ZV_7051__GP1      2021,140,16:00:00.000000 2021,140,16:00:00.000000 -0   1
ZV_7051__GP1      2021,140,16:30:00.000000 2021,140,16:30:00.000000 -0   1
ZV_7051__GP1      2021,140,17:00:00.000000 2021,140,17:00:00.000000 -0   1
ZV_7051__GP1      2021,140,17:30:00.000000 2021,140,17:30:00.000000 -0   1
ZV_7051__GP1      2021,140,18:00:00.000000 2021,140,18:00:00.000000 -0   1
ZV_7051__GP1      2021,140,18:30:00.000000 2021,140,18:30:00.000000 -0   1
ZV_7051__GP1      2021,140,19:00:00.000000 2021,140,19:00:00.000000 -0   1
ZV_7051__GP1      2021,140,19:30:00.000000 2021,140,19:30:00.000000 -0   1
ZV_7051__GP1      2021,140,20:00:00.000000 2021,140,20:00:00.000000 -0   1
ZV_7051__GP1      2021,140,20:30:00.000000 2021,140,20:30:00.000000 -0   1
ZV_7051__GP1      2021,140,21:00:00.000000 2021,140,21:00:00.000000 -0   1
ZV_7051__GP1      2021,140,21:30:00.000000 2021,140,21:30:00.000000 -0   1
ZV_7051__GP1      2021,140,22:00:00.000000 2021,140,22:00:00.000000 -0   1
ZV_7051__GP1      2021,140,22:30:00.000000 2021,140,22:30:00.000000 -0   1
ZV_7051__GP1      2021,140,23:00:00.000000 2021,140,23:00:00.000000 -0   1
ZV_7051__GP1      2021,140,23:30:00.000000 2021,140,23:30:00.000000 -0   1

This PR seems to be working as we would expect, and once a unit test is added to the PR is should be merged.

hrotman-pic commented 2 years ago

Resolved by #503 .