search

PIC-IRIS / PH5

Library of PH5 clients, apis, and utilities
Other
15 stars 9 forks source link

[BUG] DAS table deletion seems to be affecting sort_kef_gen #480

Closed hrotman-pic closed 2 years ago

hrotman-pic commented 3 years ago

Describe the bug Running sort_kef_gen -a on a PH5 with DAS tables that have been deleted (and not later reloaded) results in output: Traceback (most recent call last): File "/home/ph5/anaconda3/envs/ph5/bin/sort_kef_gen", line 11, in load_entry_point('ph5', 'console_scripts', 'sort_kef_gen')() File "/home/ph5/PH5/ph5/utilities/sort_kef_gen.py", line 370, in main if not read_all_das(): File "/home/ph5/PH5/ph5/utilities/sort_kef_gen.py", line 225, in read_all_das r, k = EX.ph5_g_receivers.read_das() File "/home/ph5/PH5/ph5/core/experiment.py", line 727, in read_das ignore_srm) File "/home/ph5/PH5/ph5/core/experiment.py", line 44, in check_srm_valid if 'sample_rate_multiplier_i' not in keys: TypeError: argument of type 'NoneType' is not iterable

The resulting sort.kef is empty.

The deleted DAS tables seem to be in the DAS list (e.g., looking at the PH5 with kefedit) but do not open & print errors when they are selected.

Environment (please complete the following information):

To Reproduce Run sort_kef_gen -n master.ph5 -a > sort.kef on Monahans in Working, or on 18-016_ShaleHillsCZO/post_send_fixes/to_send in Completed.

Expected behavior Can sort_kef_gen ignore (skip?) the deleted DAS tables, so it runs without error and produces a file with contents?

Additional context Running ph5tokef -n master.ph5 -D <S/N of deleted DAS> now results in an error instead of an empty return.

ph5_validate and stationXML extraction run normally and do not produce errors (checked on Monahans).

sort_kef_gen still runs normally on recent archives without deleted DAS tables.

timronan commented 2 years ago

If the DAS tables are deleted, but the indexes are in the DAS list, don't we want at least a warning to state this inconsistency so the DAS list can be cleaned up? This kind of seems like we are hiding a problem rather than dealing with it. Doesn't ph5_validate throw a warning for empty DAS tables? Is this because the DAS table does not exist but it's index is in the list?

What would need to be changed to delete the DAS index from the DAS list when the DAS tables are deleted? I guess my hesitancy with this issue is associated with potentially hiding issues instead of resolving them.

damhuonglan commented 2 years ago

@timronan I've talked about this problem and planned for another PR to solve this. PR482 is a temporary fix to return the state how it works before we tried to check sample rate multiplier. Will work on PR for deleting all traces of das table if it is removed when I have time.

timronan commented 2 years ago

Sounds good thanks for the confirmation.

hrotman-pic commented 2 years ago

Resolved by #489.