figure out what file names are

[swiggumj]

For the Apr 22 session (p2945.cimalog_20200422), the first few filenames should be:

puppi_58961_J1640+2224_0272_cal_0001.fits puppi_58961_J1640+2224_0273_0001.fits puppi_58961_J1640+2224_0274_cal_0001.fits

...and the cimalog lines we need to track to capture scan info are:

2020-Apr-22 01:59:44 WARNING from_puppi: Coherent mode Started with Scan number : 272 2020-Apr-22 02:01:38 WARNING from_puppi: Coherent mode Started with Scan number : 273 2020-Apr-22 02:21:48 WARNING from_puppi: Coherent mode Started with Scan number : 274 2020-Apr-22 02:23:38 WARNING from_puppi: Coherent mode Started with Scan number : 275 2020-Apr-22 02:46:56 WARNING from_puppi: Coherent mode Started with Scan number : 276 2020-Apr-22 02:48:46 WARNING from_puppi: Coherent mode Started with Scan number : 277 2020-Apr-22 03:09:33 WARNING from_puppi: Coherent mode Started with Scan number : 278 2020-Apr-22 03:11:23 WARNING from_puppi: Coherent mode Started with Scan number : 279

...

[dlakaplan]

OK, a few questions:

• How is the MJD determined? Start time of the run? The observation? End time? What happens if a single pulsar spans multiple MJDs?
• Any way to determine the _cal files and the others? Do they all have _0001.fits?
• How do you want to display these? What I have now seems clumsy.

  153 sec (   132 sec slewing) --> Execute PSR B1953+29   (std) for 1427s at 1410MHz at linenumber  5654 (1290 sec requested) +
  Writing to /share/obs4/usr/pulsar/p2780/puppi_58893_B1953+29_0148, /share/obs4/usr/pulsar/p2780/puppi_58893_B1953+29_0149
  86 sec (     8 sec slewing) --> Execute PSR B1953+29   (std) for 1179s at  430MHz at linenumber  6342 (1290 sec requested) -
  Writing to /share/obs4/usr/pulsar/p2780/puppi_58893_B1953+29_0150, /share/obs4/usr/pulsar/p2780/puppi_58893_B1953+29_0151

[swiggumj]

All very good questions. Short answer -- I need to check on all of this in more detail.

There should be a pair of cal/non-cal (pulsar) scans for each source (invoked by PULSARON), but there are often multiple files associated with each scan (therefore not all files end with _0001.fits, but this number is incremented by the number of files.

I think displaying the filename alone would be plenty. Files are written to individual puppi gpus, so this is a different directory structure entirely.

[swiggumj]

I put this on Slack, but just for posterity...

Michael and Paul confirmed that MJD is set at the beginning of a scan, then remains the same throughout.

Getting incremental file numbers is not straightforward. Nate was thinking this output would be useful for feeding directly into the script he uses to combine files from different gpus, so if that's the case, I think we don't really care about how many files there are for a given scan, just that one or more files exist. I can think about this a bit more, but I think reproducing file base names (everything before _00??.fits is good enough.

To distinguish between cal/pulsar scans, maybe we can implement something like:

if winking_cal:
    ...include _cal in filename...
else:
    ...don't...

by tracking the state of winking_cal and comparing this to from_puppi:...Scan number.

[dlakaplan]

### Report for: examples/logs/p2945.cimalog_20200410
### NANOGrav p2945 observation (1.9h elapsed; 1.6h observing; 4 scans)
### 2020-04-10 02:30:36 - 2020-04-10 04:22:41
   538 sec (     7 sec slewing) --> Execute PSR J1640+2224 (std) for 1371s at  430MHz at linenumber   918 (1230 sec requested) +
    Writing to puppi_58949_J1640+2224_0037_cal, puppi_58949_J1640+2224_0038
    80 sec (     8 sec slewing) --> Execute PSR J1640+2224 (std) for 1366s at 1410MHz at linenumber  1577 (1230 sec requested) +
    Writing to puppi_58949_J1640+2224_0039_cal, puppi_58949_J1640+2224_0040
   293 sec (   248 sec slewing) --> Execute PSR J1713+0747 (std) for 1366s at 1410MHz at linenumber  2236 (1230 sec requested) +
    Writing to puppi_58949_J1713+0747_0041_cal, puppi_58949_J1713+0747_0042
   135 sec (     9 sec slewing) --> Execute PSR J1713+0747 (std) for 1516s at 2030MHz at linenumber  2894 (1380 sec requested) +
    Writing to puppi_58949_J1713+0747_0043_cal, puppi_58949_J1713+0747_0044

@swiggumj this is in 68ce74c8c6cb442db2b9b05a0afcb089b8166841 close if you are happy

[swiggumj]

Looks great!