search

simonsobs / sotodlib

Simons Observatory: Time-Ordered Data processing library.
MIT License
16 stars 19 forks source link

Observations not indexing "right" #282

Closed jeffiuliano closed 2 years ago

jeffiuliano commented 2 years ago

When indexing, some periods of data streaming, which we know have data and g3 files associated with them on disk, do not end up as observations properly. The g3 is found -- generating an obs in the obs db. But something is stopping it from finding the end. So the obs has none for the stop time, and obs.files is never populated. Here are the most recent 4 exmples:

 ufm_mv15_1659106836: 2022-07-29 15:01:40.882475 -> None (None),
 ufm_mv15_1659042790: 2022-07-28 21:13:43.485029 -> None (None),
 ufm_mv15_1658964928: 2022-07-27 23:35:51.576701 -> None (None),
 ufm_mv11_1658964662: 2022-07-27 23:31:25.869729 -> None (None),
jeffiuliano commented 2 years ago

The lower two above observations have thees g3 files on disk:

'/mnt/so1/data/penn-latr/timestreams/16589/ufm_mv15/1658960516_000.g3'
'/mnt/so1/data/penn-latr/timestreams/16589/ufm_mv15/1658964928_000.g3'
kmharrington commented 2 years ago

Should all be solved when this is merged: https://github.com/simonsobs/sotodlib/pull/284