e-merlin / eMERLIN_CASA_pipeline

This is CASA eMERLIN pipeline to calibrate data from the e-MERLIN array. Please fork the repository before making any changes and read the Coding Practices page in the wiki. Please add issues with the pipeline in the issues tab.
GNU General Public License v3.0
14 stars 11 forks source link

Setting MS order with setToCasaOrder? #48

Closed jmoldon closed 7 years ago

jmoldon commented 7 years ago

Short version: don't use setToCasaOrder, it makes things even slower!

We discussed some time ago if we should apply setToCasaOrder just after importfitsidi in order to make the rest of the steps faster.

I have conducted some calibration starting from fits files and running

importfitsidi 
setToCasaOrder
fixvis
flag autocorrelations
Hanning smoothing
aoflagger + a-priori flags
split
etc...

The test was conducted on richards (12 CPU @ 3.50GHz, 64 GB of RAM). The data set if 184 GB in the original fits format.

Times to run different steps with setToCasaOrder: summary_test2_1

Times to run different steps with NO setToCasaOrder: summary_test2_3

Conclusions:

I will update the code to take out that option. Maybe leave if it there as an option.

jmoldon commented 7 years ago

Solved in e95e32fbd64b02431e6a4e08ab401e3dcbd09eb2

Now the default behaviour is not to apply setToCasaOrder in importfitsidi.