search

ratt-ru / xova

dask-ms/codex-africanus MS averaging tool
Other
0 stars 2 forks source link

Writes non-conformant dataset upon selection of only -dc CORRECTED_DATA #24

Closed bennahugo closed 3 years ago

bennahugo commented 3 years ago

Currently when selecting only to average CORRECTED_DATA the averager writes a non-conformant average measurement set with only CORRECTED_DATA in it. In this case the column has to be renamed to DATA in the output dataset similar to the CASA split task

sjperkins commented 3 years ago

Could you be more precise about what is meant by non-conformant?

Note that the DATA column is optional:

https://casa.nrao.edu/Memos/229.html#SECTION00061000000000000000

I'm closing this issue for the above reason. Please re-open if you believe this to be incorrect.

bennahugo commented 3 years ago

The defacto standard behaviour for averaging is to put CORRECTED_DATA into DATA upon the absence of the DATA column (see e.g. behaviour of split, oldsplit and mstransform). With this column missing most CASA tasks will fail. There is no way to reinitialize DATA from CORRECTED_DATA in casa (the inverse can be done by the clearcal task). It is not urgent, I'm just marking it as a problem

Edit: A typical workflow would be: transfer calibrate into corrected data, average into DATA and then do selfcal for instance back into CORRECTED_DATA. So as it stands the standard workflow cannot be implemented.

sjperkins commented 3 years ago

The defacto standard behaviour for averaging is to put CORRECTED_DATA into DATA upon the absence of the DATA column (see e.g. behaviour of split, oldsplit and mstransform). With this column missing most CASA tasks will fail. There is no way to reinitialize DATA from CORRECTED_DATA in casa (the inverse can be done by the clearcal task). It is not urgent, I'm just marking it as a problem

From memory, it's possible to use the following switch in xova to obtain the desired behaviour -dc CORRECTED_DATA:DATA

bennahugo commented 3 years ago

Oh? Ok then that is fine though.

sjperkins commented 3 years ago

Note the defaults from xova bda --help:

  -dc DATA_COLUMN [DATA_COLUMN ...], --data-column DATA_COLUMN [DATA_COLUMN ...]
                        Column(s) to average. Default CORRECTED_DATA:DATA
sjperkins commented 3 years ago

Help could be improved tho. Could you add a sentence in your current PR to mention the -dc source:target convention?

bennahugo commented 3 years ago

Sure will do

On Mon, Mar 15, 2021 at 9:49 AM Simon Perkins @.***> wrote:

Help could be improved tho. Could you add a sentence in your current PR to mention the -dc source:target convention?

— You are receiving this because you modified the open/close state. Reply to this email directly, view it on GitHub https://github.com/ska-sa/xova/issues/24#issuecomment-799196045, or unsubscribe https://github.com/notifications/unsubscribe-auth/AB4RE6W2GAJC6WY2NNL4ESDTDW3X7ANCNFSM4ZFBUSVQ .

--

Benjamin Hugo

PhD. student, Centre for Radio Astronomy Techniques and Technologies Department of Physics and Electronics Rhodes University

Junior software developer Radio Astronomy Research Group South African Radio Astronomy Observatory Black River Business Park Observatory Cape Town