search

PIC-IRIS / PH5

Library of PH5 clients, apis, and utilities
Other
15 stars 9 forks source link

[FEATURE] ph5tostationxml utilize additional fields from Array_t #541

Open hrotman-pic opened 2 weeks ago

hrotman-pic commented 2 weeks ago

Is your feature request related to a problem? Please describe. The current ph5tostationxml is calling on deploy_time/epoch_l and pickup_time/epoch_l, which are whole seconds. Microseconds are stored in a different field in the array table, so if that field is populated the current stationXML is (slightly) incorrect. Applies to multiple experiments, likely more than half the repository as defined by number of channels. The DAS SN for nodal experiments is not the actual physical unit serial number, but lineXstation. The physical unit serial number is included in the field description_s. An example description_s: "DAS: 1X1, Node ID: 3034".

Describe the solution you'd like ph5tostationxml call on deploy_time/micro_seconds_i and pickup_time/micro_seconds_i if they are not null and include in start and end time in the stationXML. Also, use description_s if not not null, but uncertain which field in the stationXML schema can be used to list this information in the output stationXML file. If any of these fields are null, ph5tostationxml should proceed without warning or error.

Additional context With the planned migration of PH5 to TileDB, it is preferable to include as much information as possible from Array_t in the stationXML since the PH5 tables will no longer be part of the repository and may be deleted at some point in the (far?) future.

hrotman-pic commented 1 week ago

Note: split on pull request with different test cases.

Microseconds not being in the stationxml: bug.

Adding description_s: enhancement.

hrotman-pic commented 1 week ago

Example case where microseconds are present in Array_t but not in the stationXML and the web service tools can access less data as a result: PH5 18-028, network 4M.2018, station 201.

Server-side availability (the end times exactly match the Array_t pickup time):

n s l c q sample-rate earliest latest

4M 201 -- DP1 250.0 2018-07-21T11:37:39.052000Z 2018-07-23T01:37:39.052000Z 4M 201 -- DP1 250.0 2018-07-23T02:55:08.864000Z 2018-08-24T21:51:38.947000Z 4M 201 -- DP1 250.0 2018-07-21T11:37:39.052000Z 2018-07-23T01:37:39.052000Z 4M 201 -- DP1 250.0 2018-07-23T02:55:08.864000Z 2018-08-24T21:51:38.947000Z 4M 201 -- DP2 250.0 2018-07-21T11:37:39.052000Z 2018-07-23T01:37:39.052000Z 4M 201 -- DP2 250.0 2018-07-23T02:55:08.864000Z 2018-08-24T21:51:38.947000Z 4M 201 -- DP2 250.0 2018-07-21T11:37:39.052000Z 2018-07-23T01:37:39.052000Z 4M 201 -- DP2 250.0 2018-07-23T02:55:08.864000Z 2018-08-24T21:51:38.947000Z 4M 201 -- DPZ 250.0 2018-07-21T11:37:39.052000Z 2018-07-23T01:37:39.052000Z 4M 201 -- DPZ 250.0 2018-07-23T02:55:08.864000Z 2018-08-24T21:51:38.947000Z 4M 201 -- DPZ 250.0 2018-07-21T11:37:39.052000Z 2018-07-23T01:37:39.052000Z 4M 201 -- DPZ 250.0 2018-07-23T02:55:08.864000Z 2018-08-24T21:51:38.947000Z

Server-side mseed for the last day: msi -S *08-24* DCC|2024,248 4M|201||DP1|2018,236,00:00:00.000000|2018,236,21:51:38.944000||250|19674737|||||||2024,248 4M|201||DP2|2018,236,00:00:00.000000|2018,236,21:51:38.944000||250|19674737|||||||2024,248 4M|201||DPZ|2018,236,00:00:00.000000|2018,236,21:51:38.944000||250|19674737|||||||2024,248

The difference of 1 sample between availability and mseed output was discussed and okayed during review of PR505.

PH5WS availability:

Network Station Location Channel Quality SampleRate Earliest Latest

4M 201 -- DP1 D 250.0 2018-07-21T11:37:39.050000Z 2018-07-23T01:37:39.050000Z 4M 201 -- DP1 D 250.0 2018-07-23T02:55:08.860000Z 2018-08-24T21:51:38.000000Z 4M 201 -- DP2 D 250.0 2018-07-21T11:37:39.050000Z 2018-07-23T01:37:39.050000Z 4M 201 -- DP2 D 250.0 2018-07-23T02:55:08.860000Z 2018-08-24T21:51:38.000000Z 4M 201 -- DPZ D 250.0 2018-07-21T11:37:39.050000Z 2018-07-23T01:37:39.050000Z 4M 201 -- DPZ D 250.0 2018-07-23T02:55:08.860000Z 2018-08-24T21:51:38.000000Z

PH5WS dataselect mseed for the last day: msi -S ph5ws-dataselect_2024-09-04t14_01_41z.mseed DCC|2024,248 4M|201||DP1|2018,236,00:00:00.000000|2018,236,21:51:37.996000||250|19674500|||||||2024,248 4M|201||DP2|2018,236,00:00:00.000000|2018,236,21:51:37.996000||250|19674500|||||||2024,248 4M|201||DPZ|2018,236,00:00:00.000000|2018,236,21:51:37.996000||250|19674500|||||||2024,248