search

PIC-IRIS / PH5

Library of PH5 clients, apis, and utilities
Other
15 stars 9 forks source link

[FEATURE] Path to archive RT72A active source data #539

Closed hrotman-pic closed 1 week ago

hrotman-pic commented 2 weeks ago

Is your feature request related to a problem? Please describe. This is to archive an older active-source dataset that used PASSCAL RT72As. It is unknown if there are additional active-source RT72A experiments that could/should be archived in PH5 format. The raw data files are named xxxx.ref and 130toph5 attempts to read them but errors on the gain and bit weight, which appear to exist but have different formatting than expected. Example errors: [2020-02-17 17:16:14,587] - ph5.utilities.1302ph5 - ERROR: Warning: No total number of channels for EH packet given.Setting to 6. [2020-02-17 17:16:14,587] - ph5.utilities.1302ph5 - ERROR: *************** END *************** [2020-02-17 17:16:14,595] - ph5.utilities.1302ph5 - WARNING: Can't determine gain from gain value 'Unknown'. Exception: invalid literal for int() with base 10: 'Unkno' [2020-02-17 17:16:14,595] - ph5.utilities.1302ph5 - ERROR: Bit weight undefined. Can't convert ' '. Exception: could not convert string to float:

Describe the solution you'd like RT72A data can be added to PH5, or a solution that permits discoverable archival of this and any similar experiments. Sample data files are available.

Describe alternatives you've considered The originally planned archival path for the experiment, after encountering the above errors, was to use the fully retired ref2mseed code to convert from the raw format, then use mstoph5 and metadatatoph5. It may eventually be possible to have an alternate archival path, such as ref2mseed and then mseed to TileDB, with the metadata being added by a method to be determined. Array tables (.csv, before noven) exist for this experiment, which may enable metadata generation for an alternative solution.

Additional context The RT72A data format is similar to the RT130; my understanding is RT130 programming is based on RT72A programming. They are similar enough that the PASSOFT tool logpeek can read both of them. It is unknown if any previous version of the 130toph5 code could successfully read RT72A data. If this issue is declined, the PH5 group should discuss how to archive this and any similar experiments in a timely fashion that utilizes a reasonable number of development cycles.

hrotman-pic commented 1 week ago

Following discussion, this experiment will be archived outside of PH5.