search

NCAR / chromag-pipeline

Data processing pipeline for ChroMag
1 stars 0 forks source link

Make inventory files #2

Open mgalloy opened 2 years ago

mgalloy commented 2 years ago

There is raw data to test on in:

/export/data1/Data/ChroMag/raw

on our incoming data server.

mgalloy commented 1 year ago

The basic inventory file looks like:

20220304$ cat 20220304.chromag.inventory.txt
20220304T223344.795Z.fits   Scientific   617.505 nm
20220304T222903.841Z.fits   Scientific   617.565 nm
20220304T222852.892Z.fits   Scientific   617.505 nm
20220304T222859.314Z.fits   Scientific   617.540 nm
20220304T222855.693Z.fits   Scientific   617.520 nm
20220304T222940.393Z.fits   Scientific   617.580 nm
20220304T223831.821Z.fits   Scientific   617.565 nm
20220304T223420.909Z.fits   Scientific   617.595 nm
20220304T223823.571Z.fits   Scientific   617.520 nm
...

It will now be easy to add other columns to this file.

mgalloy commented 1 year ago

@dwijn @jburkepile @bberkeyU What columns would be useful for the inventory file?

mgalloy commented 1 year ago

I added scan index (SCAN_I and SCAN_N):

20220304$ cat 20220304.chromag.inventory.txt
20220304T223344.795Z.fits   Scientific   617.505 nm       2      22
20220304T222903.841Z.fits   Scientific   617.565 nm      14      22
20220304T222852.892Z.fits   Scientific   617.505 nm       2      22
20220304T222859.314Z.fits   Scientific   617.540 nm       9      22
20220304T222855.693Z.fits   Scientific   617.520 nm       5      22
20220304T222940.393Z.fits   Scientific   617.580 nm      17      22
20220304T223831.821Z.fits   Scientific   617.565 nm      14      22
20220304T223420.909Z.fits   Scientific   617.595 nm      20      22
20220304T223823.571Z.fits   Scientific   617.520 nm       5      22
...