Shouldn't there be an option of manual flagging before the AOflagger step?

e-merlin / eMERLIN_CASA_pipeline

This is CASA eMERLIN pipeline to calibrate data from the e-MERLIN array. Please fork the repository before making any changes and read the Coding Practices page in the wiki. Please add issues with the pipeline in the issues tab.

GNU General Public License v3.0

14 stars 11 forks source link

Shouldn't there be an option of manual flagging before the AOflagger step? #67

Closed varenius closed 6 years ago

varenius commented 6 years ago

I wonder: Shouldn't there be an option of manual flagging before the AOflagger step? Say that I know that one antenna is completely rubbish and should be removed. It would be good, I think, to flag and remove this antenna completely even before the AOflagger step.

jmoldon commented 6 years ago

The main problem is that I still don't see a way for aoflagger to work on partially flagged data. I think it will still run and write its own flags forgetting previous flags (at some point I thought I had seen a parameter to avoid that, but I cannot find it). So I don't think it will help in performance because all data will be read anyway. Also, with the current method, aoflagger is only run at the beginning, so the rest of casa commands can be tracked with the flagmanager in casa.

Related to that, I don't know if there is a casa command to reset all flags to 0 (regardless of who flagged the data). We can do that with the tb toolkit, modifying the whole FLAG column, but I still hope there is a better way that allows you to select what part of the data to completely unflag.

jradcliffe5 commented 6 years ago

Hi all,

Related to that, I don't know if there is a casa command to reset all flags to 0 (regardless of who flagged the data). We can do that with the tb toolkit, modifying the whole FLAG column, but I still hope there is a better way that allows you to select what part of the data to completely unflag.

The CASA task flagdata has the mode ‘unflag’ so you can just run this on the data with everything else default in order to remove the flags. I’m not too sure whether it removes AOflags, we should probably test this?

jmoldon commented 6 years ago

Two things explaining how aoflagger works:

aoflagger will not use information on previous flags. It just takes all the visibilities and produces new flags, but always considers the flagged data during flagging.
There is a way to keep previous flags after it runs. At the end of the strategy you need to add a new action:
```
  <action type="SetFlaggingAction">
    <new-flagging>6</new-flagging>
  </action>
```
But because it will ignore previous flags (= will use all flagged and unflagged data) for flagging, it does not matter if it is run before or after any other flagging step.