askap-craco / CELEBI

The CRAFT Effortless Localisation and Enhanced Burst Inspection Pipeline
MIT License
4 stars 2 forks source link

RFI correlations fail at difx_to_fits because some cards get 0 vis (FITLD returned '2') #472

Open d-r-scott opened 1 year ago

d-r-scott commented 1 year ago

difx_to_fits work directory for 220610's RFI correlation:

$ cd /fred/oz002/askap/craft/craco/processing/work/220610/d1/319302e8d973f571743dc0de6a2158
$ ll
...
-rw-r--r-- 1 dscott oz002 337106880 Jul  6 16:01 CRAFT_CARD1.FITS
-rw-r--r-- 1 dscott oz002 253045440 Jul  6 16:01 CRAFT_CARD2.FITS
-rw-r--r-- 1 dscott oz002  45385920 Jul  6 16:01 CRAFT_CARD3.FITS
-rw-r--r-- 1 dscott oz002    861120 Jul  6 16:02 CRAFT_CARD4.FITS
-rw-r--r-- 1 dscott oz002    861120 Jul  6 16:02 CRAFT_CARD5.FITS
-rw-r--r-- 1 dscott oz002    861120 Jul  6 16:02 CRAFT_CARD6.FITS
-rw-r--r-- 1 dscott oz002    861120 Jul  6 16:02 CRAFT_CARD7.FITS
...

Going to c4_f2:

$ cd /fred/oz002/askap/craft/craco/processing/work/220610/a1/5a6e69422b719df3b1770cdd7815e0/c4_f2
$ cat tscrunch_log.txt     #run on reference correlation
Read 16192 records in, wrote 8096 back out after averaging
The difference (in seconds) between the chosen header time and the mean MJD in the input data was 0.0
All visibility times were set to MJD 59740.93523352083

$ cat tscrunch_log2.txt     #run on current correlation
Read 0 records in, wrote 0 back out after averaging
The difference (in seconds) between the chosen header time and the mean MJD in the input data was 5161616804.1762
All visibility times were set to MJD 59740.93523352083

$ ll craftfrb_4530_old.difx
-rw-r--r-- 1 dscott oz002 0 Jul  6 15:45 DIFX_59740_080802.s0000.b0000

$ ll craftfrb_4530_tscrunch.difx
-rw-r--r-- 1 dscott oz002 0 Jul  6 15:48 DIFX_59740_080802.s0000.b0000

$ ll craftfrb_4530.difx
-rw-r--r-- 1 dscott oz002 8889408 Jul  6 15:48 DIFX_59740_080802.s0000.b0000

Looks like something is being filled in, but maybe not enough? Compare to c1_f0 (reference correlation):

$ cd /fred/oz002/askap/craft/craco/processing/work/220610/83/d665fc9f4a04b21eac1ebe956dd336/c1_f0
$ ll craftfrb_2190.difx
-rw-r--r-- 1 dscott oz002 17778816 Jul  6 14:56 DIFX_59740_080802.s0000.b0000
d-r-scott commented 1 year ago

The reference correlation is tscrunched when used by the non-reference correlations, but the .difx that is passed through to difx_to_fits itself is possibly never tscrunched. Could this be the problem?