PIC-IRIS / PH5

Library of PH5 clients, apis, and utilities
Other
15 stars 9 forks source link

extend time when there is overlap for ph5availability #505

Closed damhuonglan closed 1 year ago

damhuonglan commented 2 years ago

What does this PR do?

ph5api.py - get_availability():

Relevant Issues?

499

Checklist

hrotman-pic commented 1 year ago

I have tested this PR with SmartSolo experiments and it looks like ph5availability output now covers all expected times. For example, selected output from 21-002 station 8162: ph5availability -n master.ph5 --station 8162 -S -a 2 -f t 9N 8162 -- GP1 1000.0 2020-11-07T22:42:00.009000Z 2020-11-08T01:00:00.001999Z 9N 8162 -- GP1 1000.0 2020-11-08T01:00:00.002000Z 2020-11-08T02:30:00.004999Z

I am in the process of testing other cases in PH5 experiments: Fairfield, Fairfield multiple deployment, RT130, Texan, and rolling Texan and need to finish those tests before I can write a review on this PR that is as correct and fully informed as possible.

hrotman-pic commented 1 year ago

Here are the example availability output files for the multiple deployment Fairfield experiments I mentioned. All queries were in the format: ph5availability -n master.ph5 --station sta_id -S -a 2 -f t > outfile.txt

I use examples from two experiments, 18-027 and 19-010.

I have included availability output on master and this PR for three stations from each experiment: two stations with output that is different on master and this PR, and one station that is the same. I have also included the array table as a csv for the example stations but I can upload the full array tables too.. All of these files are output from the PH5 experiment as it currently exists on the PIC server.

18-027_array1_example_stns.csv 18-027_avail1101.txt 18-027_avail1103.txt 18-027_avail1212.txt 18-027_PR505_avail1101.txt 18-027_PR505_avail1103.txt 18-027_PR505_avail1212.txt 19-010_arrays_example_stns.csv 19-010_avail1212.txt 19-010_avail1511.txt 19-010_avail1513.txt 19-010_PR505_avail1212.txt 19-010_PR505_avail1511.txt 19-010_PR505_avail1513.txt

timronan commented 1 year ago

Test case associated with experiment 22-030:

Without this update 30 minutes of data are missing: This is the output from the Main branch:

time ph5availability -n master.ph5 -p . -s 2022-04-18T00:00:00 -e 2022-04-18T00:30:00 --station 02118 -a 2

#n s      l  c   q                    earliest                      latest
4Q 02118  -- DP1   2022-04-18T00:00:00.000000Z 2022-04-18T00:00:00.096000Z
4Q 02118  -- DP2   2022-04-18T00:00:00.000000Z 2022-04-18T00:00:00.096000Z
4Q 02118  -- DPZ   2022-04-18T00:00:00.000000Z 2022-04-18T00:00:00.096000Z

real    0m1.522s
user    0m1.313s
sys 0m0.819s

With the update, all of the data are there: This is a output from the i499_ph5availability branch:

time ph5availability -n master.ph5 -p . -s 2022-04-18T00:00:00 -e 2022-04-18T00:30:00 --station 02118 -a 2

#n s      l  c   q                    earliest                      latest
4Q 02118  -- DP1   2022-04-18T00:00:00.000000Z 2022-04-18T00:30:00.000000Z
4Q 02118  -- DP2   2022-04-18T00:00:00.000000Z 2022-04-18T00:30:00.000000Z
4Q 02118  -- DPZ   2022-04-18T00:00:00.000000Z 2022-04-18T00:30:00.000000Z

real    0m1.528s
user    0m1.311s
sys 0m0.831s

real, and user reports the amount of time that was required for the code to run.