search

chime-experiment / dias

A data integrity framework
https://dias.readthedocs.io/
GNU General Public License v3.0
2 stars 0 forks source link

feat(dataset_analyzer): improve logging #188

Closed nritsche closed 3 years ago

nritsche commented 3 years ago 
[2021-01-20 00:36:04,563] dias[dataset]: Found 1 acqws in flags files
[2021-01-20 00:36:04,630] dias[dataset]: Now processing acquisition /mnt/gong/staging/20210101T000000Z_chime_flaginput (7 files)
[2021-01-20 00:36:20,848] dias[dataset]: Flags don't match for dataset 486a4c7d20d88785bbbf2fdf92e3b517 (b'flaginput_20210112T131136.172720Z_rfi') between the following files:
- '/mnt/gong/staging/20201223T011000Z_chimestack_corr/01766367_0000.h5'
- '/mnt/gong/staging/20210101T000000Z_chime_flaginput/' 
[2021-01-20 00:36:20,848] dias[dataset]: Flags in 1/12 frames mismatch.
[2021-01-20 00:36:20,848] dias[dataset]: Flags for 1 inputs mismatch:
[2021-01-20 00:36:20,856] dias[dataset]: Input (1436, 'FCC040300') mismatches in 1/12 frames
[2021-01-20 00:36:20,856] dias[dataset]: The input is only flagged in the dataset state.
anjakefala commented 3 years ago

This is so cool! chef's kiss

nritsche commented 3 years ago

While you are there, you can also clean up the tracker logging that you find less helpful.

Let's do that separately, though.