o-smirnov
commented
3 years ago Oh, and when I set --weight-column None, I get cosmologically large chi-sq values. This MS just keeps on giving (bugs).
Closed o-smirnov closed 3 years ago
o-smirnov
commented
3 years ago Oh, and when I set --weight-column None, I get cosmologically large chi-sq values. This MS just keeps on giving (bugs).
o-smirnov
commented
3 years ago The latter is due to https://github.com/ratt-ru/CubiCal/issues/417.
The former seems to be fixed by the https://github.com/ratt-ru/CubiCal/tree/mk-polcal debugging spree.
o-smirnov
commented
3 years ago Nope, false dawn! Still getting crap in the output. Suspect it's the re-emergence of https://github.com/casacore/python-casacore/issues/130.
The scenario only arises with the D-solve parset. My theory is that the difference is, in this scenario we have one very large tile, due to timeint being infinite, and there being a fair chunk of 1939 data in the MS. (By contrast, the X-solve data is one small scan on 3C286. And when I apply the D and X solutions, I use smaller chunks.)
When I check in the debugger, the data & flags in memory look consistent (I check for high unflagged amplitudes, i.e. (data>100)&~flag, this returns nothing).
However shadems still plots crap as per https://github.com/ratt-ru/CubiCal/issues/415#issue-745463870, and trying to look at the data in the table with python-casacore, I run into https://github.com/casacore/python-casacore/issues/130. So I don't trust the reading or the writing at this point.
o-smirnov
commented
3 years ago Yes, the above guess is correct! With the fix for #424 in place, corrected data post-solve looks perfectly fine:
Note that this is better than the second plot in https://github.com/ratt-ru/CubiCal/issues/415#issue-745463870. So the leakage solutions are better, no crazy channels. Probably makes sense: https://github.com/casacore/python-casacore/issues/130 would have affected the input data being read too, so the previous solutions would have been off.
How has radio astronomy ever worked? :scream:
o-smirnov
commented
3 years ago With the casacore bug worked around, the leakage solutions are a thing of beauty:
| before | after |
|---|---|
 |
 |
Running a D (leakage) solution with --data-rebin-time 8 (because the solint is infinite in time, and there's just a bit too much data to load raw), I get perfectly sensible-looking solutions and chi-squares, but a horrible corrected data:
Rerunning the same solutions with apply-only (and no rebinning), things look fine:
Since in rebin mode the output data is meant to be simply broadcast out over the rebin, this is a sign of some arithmetic bug...
Run is at
/net/simon/projects/OldDevils/cc-polcal/solve-D.cc-out.5.