ACES-CMZ / reduction_ACES

Reduction scripts and tools for ACES
https://worldwidetelescope.org/webclient/?wtml=https://data.rc.ufl.edu/pub/adamginsburg/ACES/mosaics/mosaics.wtml
15 stars 12 forks source link

Array combination for CS(2-1) (12m + 7m + TP) #339

Open pyhsiehATalma opened 1 year ago

pyhsiehATalma commented 1 year ago

This is general issue for tracking the 12m+7m+TP combination for the CS(2-1) line data.

Update: 2024 Jan 31 12m+7m+TP 41 fields combination.

Proposal ID | Obs ID | 7m MOUS ID | 12m MOUS ID | issue | Note -- | -- | -- | -- | -- | -- cmz01 | a | X1590_X30ac | X1590_X30aa | #242 |   cmz02 | b | X15b4_X37 | X15a0_Xa6 | #150 | imcontsub was applied to CS2-1 cube (-300~300 km/s), the continuum subtration looks fine. cmz03 | c | X15a0_Xae | X15a0_Xac | #256 |   cmz04 | d | X15a0_Xb4 | X15a0_Xb2 | #247 |   cmz05 | e | X15b4_X39 | X15a0_Xb8 | #147 |   cmz06 | f | X15a0_Xc0 | X15a0_Xbe | #266 | imcontsub was applied to CS2-1 cube (-300~300 km/s), the continuum subtration looks fine. cmz07 | g | X15a0_Xc6 | X15a0_Xc4 | #134 |   cmz08 | h | X15a0_Xcc | X15a0_Xca | #142 |   cmz09 | i | X15b4_Xc5 | X15a0_Xd0 | #138 |   cmz10 | j | X15a0_Xd8 | X15b4_X3d | #117 |   cmz11 | k | X15a0_Xde | X15a0_Xdc | #152 |   cmz12 | l | X15a0_Xe4 | X15a0_Xe2 | #137 |   cmz13 | m | X15b4_X43 | X15b4_X41 | #118 | Sgr B2, imcontsub was applied to 12m cmz14 | n | X15a0_Xf0 | X15a0_Xee | #231 | size mitigated, need re-run pipeline cmz15 | o | X15a0_Xf6 | X15a0_Xf4 | 272 |   cmz16 | p | X15a0_Xfc | X15a0_Xfa | #149 |   cmz17 | q | X15a0_X102 | X15a0_X100 | #109 |   cmz18 | r | X15a0_X108 | X15a0_X106 | #263 | imcontsub was applied to CS2-1 cube (-300~300 km/s), the continuum subtration looks fine. cmz19 | s | X15a0_X10e | X15a0_X10c | #264 | divergence, i am doing re-imaing with full cube. cmz20 | t | X15a0_X114 | X15a0_X112 | #113 |   cmz21 | u | X15a0_X11a | X15a0_X118 | #241 |   cmz22 | v | X15a0_X120 | X15a0_X11e | #235 |   cmz23 | w | X15a0_X126 | X15a0_X124 | #124 |   cmz24 | x | X15a0_X12c | X15a0_X12a | #125 |   cmz25 | y | X15a0_X132 | X15a0_X130 | #40 |   cmz26 | z | X15a9_X12d3 | X15a0_X136 | #131 |   cmz27 | aa | X15a0_X13e | X15a0_X13c | #251 | imcontsub was applied to full spw 33, the continuum subtration looks fine. cmz28 | ab | X15a0_X144 | X15a0_X142 | #129 |   cmz29 | ac | X15a0_X14a | X15a0_X148 | #232 |   cmz30 | ad | X15a0_X150 | X15a0_X14e | #28 |   cmz31 | ae | X15a0_X156 | X15a0_X154 | #132 |   cmz32 | af | X15a0_X15c | X15a0_X15a | #255 |   cmz33 | ag | X15a0_X162 | X15a0_X160 | #136 |   cmz34 | ah | X15a0_X168 | X15a0_X166 | #238 |   cmz35 | ai | X15a0_X16e | X15a0_X16c | #130 |   cmz36 | aj | X15b4_X47 | X15a0_X172 | #123 |   cmz37 | ak | X15a9_X12d5 | X15b4_X4b | #120 |   cmz38 | al | X15a0_X180 | X15a0_X17e | #126 |   cmz39 | am | X15a0_X186 | X15a0_X184 | #259 | divergence, i am doing re-imaing with CS(2-1) cube (-300~300 km/s) cmz40 | an | X15a0_X18c | X15a0_X18a | #133 |   cmz41 | ao | X15a0_X192 | X15a0_X190 | #41 | 12m divergence at channel 645, low level and at edge, mayby ok cmz42 | ap | X15a0_X198 | X15a0_X196 | #43 |   cmz43 | aq | X15a0_X19e | X15a0_X19c | #140 | Sgr A*, divergence, imcontsub was applied to 12m, i am also doing re-imaing with CS(2-1) cube (-300~300 km/s) cmz44 | ar | X15b4_X4f | X15a0_X1a2 | #114 |   cmz45 | as | X15a0_X1aa | X15a0_X1a8 | #145 |   Jan 18 7m+TP 45m fields combination. Files are under /upload/Mosaic-CS2-1/ACA_TP_Combination**, - Combined cubes of individual fields: /Individual Cubes - MOM8 of individual fields: /Individual MOM8 - MOM0 of individual fields: /Individual MOM0 - Mosaic combined cube and moments: /Mosaic Note that the individual combined cubes were created using the full cube, while the mosaic combined cube was generated specifically for the CS(2-1) line with a velocity range of -500 to +500 km/s. Proposal ID | Obs ID | Name | vel. offset | 7m MOUS ID | issue | Divergence | Using Product | Residual Cont. | TP MOUS ID | issue -- | -- | -- | -- | -- | -- | -- | -- | -- | -- | -- cmz01 | a | - | 40 | X1590_X30ac | #214 | no |   |   | X1590_X30ae | #215 cmz02 | b | Sgr B2 | 40 | X15b4_X37 | #162 | no |   | yes | X15a0_Xaa | #252 cmz03 | c | - | 0 | X15a0_Xae | #14 | yes | yes |   | X15a0_Xb0 | #175 cmz04 | d | - | 30 | X15a0_Xb4 | #228 | ch=597-602, 613 | yes |   | X15a0_Xb6 | #227 cmz05 | e | - | -30 | X15b4_X39 | #168 | no |   |   | X15a0_Xbc | #285 cmz06 | f | - | 0 | X15a0_Xc0 | #254 | no |   | yes | X15a0_Xc2 | #315 cmz07 | g | - | 30 | X15a0_Xc6 | #161 | no |   |   | X15a0_Xc8 | #197 cmz08 | h | - | 0 | X15a0_Xcc | #151 | no |   |   | X15a0_Xce | #177 cmz09 | i | - | 30 | X15b4_Xc5 | #191 | no |   | yes | X15a0_Xd4 | #187 cmz10 | j | - | 30 | X15a0_Xd8 | #181 | no |   |   | X15b4_X3f | #324 cmz11 | k | - | 30 | X15a0_Xde | #182 | no |   |   | X15a0_Xe0 | #193 cmz12 | l | - | 0 | X15a0_Xe4 | #153 | no |   |   | X15a0_Xe6 | #224 cmz13 | m | - | 30 | X15b4_X43 | #110 | no |   | yes | X15b4_X45 | #170 cmz14 | n | - | 30 | X15a0_Xf0 | #225 | no |   |   | X15a0_Xf2 | #226 cmz15 | o | - | 0 | X15a0_Xf6 | #257 | no |   | yes | X15a0_Xf8 | #278 cmz16 | p | - | 0 | X15a0_Xfc | #148 | no |   | yes | X15a0_Xfe | #183 cmz17 | q | - | 0 | X15a0_X102 | #171 | no |   |   | X15a0_X104 | #284 cmz18 | r | - | 30 | X15a0_X108 | #240 | no |   | yes | X15a0_X10a | #246 cmz19 | s | - | 0 | X15a0_X10e | #243 | no |   |   | X15a0_X110 | #262 cmz20 | t | - | 0 | X15a0_X114 | #8 | ch=701-703 |   |   | X15a0_X116 | #172 cmz21 | u | - | 30 | X15a0_X11a | #216 | no |   |   | X15a0_X11c | #217 cmz22 | v | - | 30 | X15a0_X120 | #218 | no |   | yes | X15a0_X122 | #219 cmz23 | w | - | 30 | X15a0_X126 | #144 | no |   |   | X15a0_X128 | #173 cmz24 | x | - | 0 | X15a0_X12c | #135 | no |   |   | X15a0_X12e | #164 cmz25 | y | - | 0 | X15a0_X132 | #4 | no |   |   | X15a0_X134 | #27 cmz26 | z | - | 40 | X15a9_X12d3 | #115 | no |   |   | X15a0_X13a | #236 cmz27 | aa | - | 0 | X15a0_X13e | #237 | no |   | yes | X15a0_X140 | #229 cmz28 | ab | - | 0 | X15a0_X144 | #141 | no |   |   | X15a0_X146 | #167 cmz29 | ac | - | 0 | X15a0_X14a | #220 | no |   |   | X15a0_X14c | #221 cmz30 | ad | - | 0 | X15a0_X150 | #5 | no |   |   | X15a0_X152 | #25 cmz31 | ae | - | 0 | X15a0_X156 | #186 | ch=652,653,655 | yes |   | X15a0_X158 | #180 cmz32 | af | - | 0 | X15a0_X15c | #244 | no |   | yes | X15a0_X15e | #271 cmz33 | ag | - | 0 | X15a0_X162 | #174 | no |   |   | X15a0_X164 | #190 cmz34 | ah | - | 30 | X15a0_X168 | #230 | ch=596, 615, 617 | yes |   | X15a0_X16a | #233 cmz35 | ai | - | 0 | X15a0_X16e | #143 | no |   |   | X15a0_X170 | #169 cmz36 | aj | - | -30 | X15b4_X47 | #119 | no |   |   | X15b4_X49 | #239 cmz37 | ak | - | 30 | X15a9_X12d5 | #116 | no |   |   | X15b4_X4d | #121 cmz38 | al | - | 0 | X15a0_X180 | #139 | no |   |   | X15a0_X182 | #166 cmz39 | am | - | 30 | X15a0_X186 | #265 | no |   | yes | X15a0_X188 | #245 cmz40 | an | - | 30 | X15a0_X18c | #222 | no |   |   | X15a0_X18e | #184 cmz41 | ao | Brick | 0 | X15a0_X192 | #6 | ch=631 | need re-imaging |   | X15a0_X194 | #29 cmz42 | ap | - | 0 | X15a0_X198 | #158 | no |   |   | X3577_X6c5 | #325 cmz43 | aq | Sgr A* | 0 | X15a0_X19e | #33 | no |   | yes | X15a0_X1a0 | #223 cmz44 | ar | - | 40 | X15b4_X4f | #159 | no |   |   | X15a0_X1a6 | #317 cmz45 | as | - | 0 | X15a0_X1aa | #165 | yes, ch=616 | yes |   | X15a0_X1ac | #322 **2023 July 18/December 28:** 12m+7m+tp 37 fields mom8 and 7m+tp 44 fields mom8 maps. note that the 0, 1, 2... is not the field number. 37 Fields Proposal ID | Obs ID |   | 7m MOUS ID | 12m MOUS ID |   | TP MOUS ID -- | -- | -- | -- | -- | -- | -- cmz01 | a | 0 | X1590_X30ac | X1590_X30aa |   | X1590_X30ae cmz02 | b | 1 | X15b4_X37 | X15a0_Xa6 |   | X15a0_Xaa cmz04 | d | 2 | X15a0_Xb4 | X15a0_Xb2 |   | X15a0_Xb6 cmz05 | e | 3 | X15b4_X39 | X15a0_Xb8 |   | X15a0_Xbc cmz06 | f | 4 | X15a0_Xc0 | X15a0_Xbe |   | X15a0_Xc2 cmz07 | g | 5 | X15a0_Xc6 | X15a0_Xc4 |   | X15a0_Xc8 cmz08 | h | 6 | X15a0_Xcc | X15a0_Xca |   | X15a0_Xce cmz09 | i | 7 | X15b4_Xc5 | X15a0_Xd0 |   | X15a0_Xd4 cmz10 | j | 8 | X15a0_Xd8 | X15b4_X3d |   | X15b4_X3f cmz11 | k | 9 | X15a0_Xde | X15a0_Xdc |   | X15a0_Xe0 cmz12 | l | 10 | X15a0_Xe4 | X15a0_Xe2 |   | X15a0_Xe6 cmz14 | n | 11 | X15a0_Xf0 | X15a0_Xee |   | X15a0_Xf2 cmz15 | o | 12 | X15a0_Xf6 | X15a0_Xf4 |   | X15a0_Xf8 cmz16 | p | 13 | X15a0_Xfc | X15a0_Xfa |   | X15a0_Xfe cmz17 | q | 14 | X15a0_X102 | X15a0_X100 |   | X15a0_X104 cmz21 | u | 15 | X15a0_X11a | X15a0_X118 |   | X15a0_X11c cmz22 | v | 16 | X15a0_X120 | X15a0_X11e |   | X15a0_X122 cmz23 | w | 17 | X15a0_X126 | X15a0_X124 |   | X15a0_X128 cmz24 | x | 18 | X15a0_X12c | X15a0_X12a |   | X15a0_X12e cmz25 | y | 19 | X15a0_X132 | X15a0_X130 |   | X15a0_X134 cmz26 | z | 20 | X15a9_X12d3 | X15a0_X136 |   | X15a0_X13a cmz27 | aa | 21 | X15a0_X13e | X15a0_X13c |   | X15a0_X140 cmz28 | ab | 22 | X15a0_X144 | X15a0_X142 |   | X15a0_X146 cmz29 | ac | 23 | X15a0_X14a | X15a0_X148 |   | X15a0_X14c cmz30 | ad | 24 | X15a0_X150 | X15a0_X14e |   | X15a0_X152 cmz31 | ae | 25 | X15a0_X156 | X15a0_X154 |   | X15a0_X158 cmz33 | ag | 26 | X15a0_X162 | X15a0_X160 |   | X15a0_X164 cmz34 | ah | 27 | X15a0_X168 | X15a0_X166 |   | X15a0_X16a cmz35 | ai | 28 | X15a0_X16e | X15a0_X16c |   | X15a0_X170 cmz36 | aj | 29 | X15b4_X47 | X15a0_X172 |   | X15b4_X49 cmz37 | ak | 30 | X15a9_X12d5 | X15b4_X4b |   | X15b4_X4d cmz38 | al | 31 | X15a0_X180 | X15a0_X17e |   | X15a0_X182 cmz41 | ao | 32 | X15a0_X192 | X15a0_X190 |   | X15a0_X194 cmz42 | ap | 33 | X15a0_X198 | X15a0_X196 |   | X3577_X6c5 cmz43 | aq | 34 | X15a0_X19e | X15a0_X19c |   | X15a0_X1a0 cmz44 | ar | 35 | X15b4_X4f | X15a0_X1a2 |   | X15a0_X1a6 cmz45 | as | 36 | X15a0_X1aa | X15a0_X1a8 |   | X15a0_X1ac
pyhsiehATalma commented 1 year ago

The testing script is to feather 12m data with existing 7m+TP cubes (made in another script). The next step is to combine these two scripts. Basically they are doing the same thing. Note that primary beam corrected files (*.pbcor) should be used. The maps I show below used non-primary corrected one (just an error).

intensity peak maps (moment 8) of these 25 fields are shown below. 25eb-p2 25eb-p3

25eb-p1

pyhsiehATalma commented 1 year ago

Mosaic intensity peak map of 25 fields is shown below. Upper panel shows 12m+7m+TP, lower panel shows 7m+TP. whole-25eb mosaic m8 v2 image fits-7m+tp m8 image fits-image-2023-05-05-13-25-30

Zoomed-in maps are shown below. Left panel shows 12m+7m+TP, right panel shows 7m+TP.

SgrB-25eb mosaic m8 v2 image fits-7m+tp m8 image fits-image-2023-05-05-13-29-02 SgrB2-25eb mosaic m8 v2 image fits-7m+tp m8 image fits-image-2023-05-05-13-30-57 sgrC-25eb mosaic m8 v2 image fits-7m+tp m8 image fits-image-2023-05-05-13-27-47

pyhsiehATalma commented 1 year ago

just a note: recording hardisk memory required and time for calculation.

pyhsiehATalma commented 1 year ago

Trouble shooting 1) If you see this error when running casa task feather, it means one of image has multiple beam information in the header. Sometimes it happens, and the solution is to use casa task imsmooth to restore the image to a common beam. imsmooth(imagename='input.image', kernel='c',outfile='out.image')

2023-07-20 04:44:12 SEVERE feather::imager::feather() Caught exception: (casacore/images/Images/ImageBeamSet.cc : 115) Failed AlwaysAssert chan >=0 && chan < Int(nchan()) && stokes >= 0 && stokes < Int(nstokes()) 2023-07-20 04:44:12 SEVERE feather::::casa Task feather raised an exception of class RuntimeError with the following message: 2023-07-20 04:44:12 SEVERE feather::imager::feather() Caught exception: (casacore/images/Images/ImageBeamSet.cc : 115) Failed AlwaysAssert chan >=0 && chan < Int(nchan()) && stokes >= 0 && stokes < Int(nstokes()) 2023-07-20 04:44:12 SEVERE feather::::casa Exception Reported: Error in feather: 2023-07-20 04:44:12 SEVERE feather::imager::feather() Caught exception: (casacore/images/Images/ImageBeamSet.cc : 115) Failed AlwaysAssert chan >=0 && chan < Int(nchan()) && stokes >= 0 && stokes < Int(nstokes()) 962.793-2023-07-20 05:25:02 SEVERE feather::imager::feather() Caught exception: (casacore/images/Images/ImageBeamSet.cc : 115) Failed AlwaysAssert chan >=0 && chan < Int(nchan()) && stokes >= 0 && stokes < Int(nstokes())

2) axes orders of two images don't match (I will copy the error messages later). Sometimes the axes orders of input images are different due to different casa version, so feather also fails. The solution is to use task imtrans to change the orders.

for example, imtrans(imagename='input.image',order='0132',out='output.image')

There are more errors, TBC.

pyhsiehATalma commented 1 year ago

12m+7m+tp 37 fields mom8 and 7m+tp 44 fields mom8 maps.

s44-s37

pyhsiehATalma commented 7 months ago

2024 Jan 17 45 Fields ACA+TP combined maps. Upper: CS(2-1) MOM0 Lower: CS(2-1) MOM8 Note: The divergence in most of the fields has been resolved; for those that could not be resolved, product images (with no divergence) were used for combination. Therefore, this 45-field map does not have divergence, except for channel 637 in field ao (X15a0_X192), which shows a slight divergence.

CS21-TP_ACA_feather-Mosaic-45field Gal image imsub moment m0-CS21-TP_ACA_feather-Mosaic-45field Gal image imsub moment m8-image-2024-01-17-11-38-33

Residual of Continuum Emission

13 Fields show continuum residual. Below I show the maps integrated from -500km/s - -400 km/s (line-free region in the CS(2-1) ACA+TP mosaic cube). Field aq, am, af, aa, o, p, v, x, m, r, i, f, b. Lower: residual of continuum emission Upper: residual of continuum emission (contours) overlaid on the CS(2-1) mom0 map. The continuum residual is weaker than CS(2-1) map.

截圖 2024-01-17 15 43 27

截圖 2024-01-17 21 20 25

Update of 12m+ACA+TP combined maps. TBC...

pyhsiehATalma commented 7 months ago

2024 Jan 24 12m+7m+TP combination, problems of the 12m data.

Missing in Mosaic | Obs ID | 12m MOUS ID | Notes | issue | Assigned To |   -- | -- | -- | -- | -- | -- | -- no included in mosaic | m | X15b4_X41 | yes and contsub issue | #118 | Nazar | Divergence fixed, constub issue remains no included in mosaic | s | X15a0_X10c | divergence, Pei-Ying is re-imaging. | #264 | Qizhou / Pei Ying (in progress) |     | aa | X15a0_X13c | divergence was fixed, but continuum is not subtracted? | #251 | Katha | imcontsub over full spw no included in mosaic | aq | X15a0_X19c | divergence at channel 638 | #140 | Dan (in progress) |   no included in mosaic | am | X15a0_X184 | there are two cubes | #259 | Dan (done) |     |   |   | Sgr_A_star_sci33.cube.I.manual.pbcor.fits |   |   | only 1917 channels, no divergence   |   |   | uid___A001_X15a0_X184.Sgr_A_star_sci33.cube.I.manual.image.pbcor |   |   | divergence around ch=626   | n | X15a0_Xee | bandwidth=1917 channel, need re-run pipeline | #231 |   |     | ad | X15a0_X14e | 5" beam | #28 |   | 2" is doable for narrow bandwidth   | l | X15a0_Xe2 | multibeam | #137 |   |     | ao | X15a0_X190 | divergence at channel 645 | #41 |   |     | f | X15a0_Xbe | contsub issue? | #266 |   | imcontsub over narrow bandwidth   | r | X15a0_X106 | contsub issue? | #263 |   | imcontsub over narrow bandwidth   | b | X15a0_Xa6 | absorption? mask may be needed for negative emission | #150 |   |  
pyhsiehATalma commented 7 months ago

moment 8 maps of 41 EB 12m+7m+TP combined maps.

41eb 41eb-2 41eb-3 41eb-4

pyhsiehATalma commented 7 months ago

I started working on mosaic of 45 fields, but note that final beam size will be 3". This is because several fields - X15a, X184, Xac have larger beam sizes (2.758"x1.964", 2.837"x1.92", 2.526"x1.738").

Also, I need to subtract the residual of the continuum on 7m and 12m if there is any, but I will leave this task for next week. Mosaic first!

I am also re-imaging on,

X15a0_X10c: full band X15a0_X19c: narrow bandwidth X15a0_X184: narrow bandwidth

pyhsiehATalma commented 7 months ago

Full CMZ 12m+7m+TP map. Upper: moment 0. Lower: moment 8. bandwidth of mosaic CS(2-1) cube: -250 km/s - 250 km/s beam: 3"

Just my personal idea: those black dots seem natural, likely due to foreground contamination, especialy in band 3. high

Remaining Issues Proposal ID Obs ID 7m MOUS ID 12m MOUS ID issue Iusse For the purpose of combination
cmz02 b X15b4_X37 X15a0_Xa6 #150 residual of continuum emission I applied imcontsub to the CS2-1 cube (-300~300 km/s), the continuum subtration looks fine.
cmz06 f X15a0_Xc0 X15a0_Xbe #266 residual of continuum emission I applied imcontsub to the CS2-1 cube, the continuum subtration looks fine.
cmz13 m X15b4_X43 X15b4_X41 #118 residual of continuum emission Sgr B2, I applied imcontsub to the CS2-1 cube (-300~300 km/s), slightly improved.
cmz14 n X15a0_Xf0 X15a0_Xee #231 size mitigation need re-run pipeline
cmz18 r X15a0_X108 X15a0_X106 #263 residual of continuum emission I applied imcontsub to the CS2-1 cube (-300~300 km/s), the continuum subtration looks fine.
cmz19 s X15a0_X10e X15a0_X10c #264 divergence I am doing re-imaing with full cube, parallel tclean is ongoing
cmz27 aa X15a0_X13e X15a0_X13c #251 continuum was not subtracted in the cube I applied imcontsub to the full spw 33, the continuum subtration looks fine.
cmz39 am X15a0_X186 X15a0_X184 #259 divergence I am doing re-imaing with the CS(2-1) cube (-300~300 km/s)
cmz41 ao X15a0_X192 X15a0_X190 #41 low-level divergence no action item
cmz43 aq X15a0_X19e X15a0_X19c #140 divergence and residual of continuum Sgr A*, I applied imcontsub to 12m, i am also doing re-imaing with CS(2-1) cube (-300~300 km/s)