search

ACES-CMZ / reduction_ACES

Reduction scripts and tools for ACES
https://worldwidetelescope.org/webclient/?wtml=https://data.rc.ufl.edu/pub/adamginsburg/ACES/mosaics/mosaics.wtml
15 stars 12 forks source link

Execution Block ID uid://A001/X15a0/X106 Sgr_A_st_r_03_TM1 #263

Open keflavich opened 2 years ago

keflavich commented 2 years ago

Sgr_A_st_r_03_TM1 uid://A001/X15a0/X106

Product Links:

Reprocessed Product Links:

keflavich commented 2 years ago

The pipeline tried to run this and failed with ValueError: No MSes found. This most likely indicates that the pipeline ran prematurely but I'm noting this.

keflavich commented 1 year ago

I removed calibrated/ today, as it was empty. The pipeline should rerun tonight.

keflavich commented 1 year ago

Re-failed:

2022-12-01 05:28:26     INFO    hifa_importdata::::casa ##########################################
2022-12-01 05:28:26     INFO    hifa_importdata::::casa ##### Begin Task: hifa_importdata    #####
2022-12-01 05:28:26     INFO    hifa_importdata::::casa hifa_importdata( vis=['../rawdata/uid___A002_Xfeb5e6_X4fa5', '../rawdata/uid___A002_Xfec857_X9f6'], session=['session_1', 'session_2'], pipelinemode='automatic', asis='SBSummary ExecBlock Antenna Station Receiver Source CalAtmosphere CalWVR CalPointing', process_caldevice=False, overwrite=False, nocopy=False, bdfflags=True, asimaging=False, lazy=False, dbservice=False, ocorr_mode='ca', createmms='false', minparang=0.0, dryrun=False, acceptresults=True )
2022-12-01 05:28:26     INFO    hifa_importdata::pipeline.infrastructure.basetask::casa Starting execution for stage 1
2022-12-01 05:28:26     INFO    hifa_importdata::pipeline.infrastructure.basetask::casa Equivalent CASA call: hifa_importdata(minparang=0.0, createmms='false', ocorr_mode='ca', dbservice=False, lazy=False, asimaging=False, bdfflags=True, nocopy=False, overwrite=False, process_caldevice=False, asis='SBSummary ExecBlock Antenna Station Receiver Source CalAtmosphere CalWVR CalPointing', session=['session_1', 'session_2'], vis=['uid___A002_Xfeb5e6_X4fa5', 'uid___A002_Xfec857_X9f6'])
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa Error executing pipeline task hifa_importdata.
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa Traceback (most recent call last):
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 640, in execute
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+            return self._handle_multiple_vis(dry_run, **parameters)
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 762, in _handle_multiple_vis
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+            single_result = self.execute(dry_run=dry_run, **parameters)
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 40, in attach_timestamp_to_results
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 120, in handle_matplotlibrc
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+            result = method(self, *args, **kwargs)
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 88, in capture
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 61, in finalise_pipeline_result
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 660, in execute
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+            result = self.prepare(**prepare_parameters)
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/h/tasks/importdata/importdata.py", line 174, in prepare
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+            raise TypeError('{!s} is of unhandled type'.format(vis))
2022-12-01 05:28:26     SEVERE  hifa_importdata::pipeline.infrastructure.basetask::casa+        TypeError: ../rawdata/uid___A002_Xfeb5e6_X4fa5 is of unhandled type
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa Task hifa_importdata raised an exception of class PipelineException with the following message: Traceback (most recent call last):
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 640, in execute
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            return self._handle_multiple_vis(dry_run, **parameters)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 762, in _handle_multiple_vis
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            single_result = self.execute(dry_run=dry_run, **parameters)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 40, in attach_timestamp_to_results
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 120, in handle_matplotlibrc
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = method(self, *args, **kwargs)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 88, in capture
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 61, in finalise_pipeline_result
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 660, in execute
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = self.prepare(**prepare_parameters)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/h/tasks/importdata/importdata.py", line 174, in prepare
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            raise TypeError('{!s} is of unhandled type'.format(vis))
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+        TypeError: ../rawdata/uid___A002_Xfeb5e6_X4fa5 is of unhandled type
2022-12-01 05:28:27     INFO    hifa_importdata::::casa Task hifa_importdata complete. Start time: 2022-12-01 00:28:25.548678 End time: 2022-12-01 00:28:26.910503
2022-12-01 05:28:27     INFO    hifa_importdata::::casa ##### End Task: hifa_importdata      #####
2022-12-01 05:28:27     INFO    hifa_importdata::::casa ##########################################
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa Exception Reported: Error in hifa_importdata: Traceback (most recent call last):
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 640, in execute
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            return self._handle_multiple_vis(dry_run, **parameters)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 762, in _handle_multiple_vis
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            single_result = self.execute(dry_run=dry_run, **parameters)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 40, in attach_timestamp_to_results
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 120, in handle_matplotlibrc
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = method(self, *args, **kwargs)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 88, in capture
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 61, in finalise_pipeline_result
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 660, in execute
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            result = self.prepare(**prepare_parameters)
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/h/tasks/importdata/importdata.py", line 174, in prepare
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+            raise TypeError('{!s} is of unhandled type'.format(vis))
2022-12-01 05:28:27     SEVERE  hifa_importdata::::casa+        TypeError: ../rawdata/uid___A002_Xfeb5e6_X4fa5 is of unhandled type
2022-12-01 05:28:27     INFO    hifa_importdata::::casa Traceback (most recent call last):
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/hifa/cli/gotasks/hifa_importdata.py", line 656, in __call__
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            _return_result_ = _hifa_importdata_t( _invocation_parameters['vis'],_invocation_parameters['session'],_invocation_parameters['pipelinemode'],_invocation_parameters['asis'],_invocation_parameters['process_caldevice'],_invocation_parameters['overwrite'],_invocation_parameters['nocopy'],_invocation_parameters['bdfflags'],_invocation_parameters['asimaging'],_invocation_parameters['lazy'],_invocation_parameters['dbservice'],_invocation_parameters['ocorr_mode'],_invocation_parameters['createmms'],_invocation_parameters['minparang'],_invocation_parameters['dryrun'],_invocation_parameters['acceptresults'] )
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/hifa/cli/hifa_importdata.py", line 112, in __call__
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            task_result = _hifa_importdata_t( _pc.document['vis'], _pc.document['session'], _pc.document['pipelinemode'], _pc.document['asis'], _pc.document['process_caldevice'], _pc.document['overwrite'], _pc.document['nocopy'], _pc.document['bdfflags'], _pc.document['asimaging'], _pc.document['lazy'], _pc.document['dbservice'], _pc.document['ocorr_mode'], _pc.document['createmms'], _pc.document['minparang'], _pc.document['dryrun'], _pc.document['acceptresults'] )
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/hifa/cli/private/task_hifa_importdata.py", line 30, in hifa_importdata2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            results = utils.execute_task(context, task_name, all_inputs)
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/h/cli/utils.py", line 86, in execute_task
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            raise exceptions.PipelineException(previous_tracebacks_as_string)
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+        pipeline.infrastructure.exceptions.PipelineException: Traceback (most recent call last):
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 640, in execute
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            return self._handle_multiple_vis(dry_run, **parameters)
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 762, in _handle_multiple_vis
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            single_result = self.execute(dry_run=dry_run, **parameters)
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 40, in attach_timestamp_to_results
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 120, in handle_matplotlibrc
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            result = method(self, *args, **kwargs)
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 88, in capture
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 61, in finalise_pipeline_result
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            result = method(self, *args, **kw)
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/infrastructure/basetask.py", line 660, in execute
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            result = self.prepare(**prepare_parameters)
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+          File "/blue/adamginsburg/adamginsburg/casa/casa-6.4.3-2-pipeline-2021.3.0.17/lib/py/lib/python3.6/site-packages/Pipeline-2021.3.0.17-py3.6.egg/pipeline/h/tasks/importdata/importdata.py", line 174, in prepare
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+            raise TypeError('{!s} is of unhandled type'.format(vis))
2022-12-01 05:28:27     INFO    hifa_importdata::::casa+        TypeError: ../rawdata/uid___A002_Xfeb5e6_X4fa5 is of unhandled type
2022-12-01 05:28:27     SEVERE  open::uid___A002_Xfeb5e6_X4fa5.ms/SYSCAL        Exception Reported: Table uid___A002_Xfeb5e6_X4fa5.ms/SYSCAL does not exist
2022-12-01 05:28:27     INFO    h_save::::casa  ##########################################
keflavich commented 1 year ago

Probably a corrupt file:

$ ls -lhrt raw/*uid___A002_Xfeb5e6_X4fa5*
total 744K
-rw-r--r-- 1 adamginsburg adamginsburg 711K Sep 22 21:10 Main.xml
-rw-r--r-- 1 adamginsburg adamginsburg  12K Sep 22 21:10 ASDM.xml
drwxr-xr-x 2 adamginsburg adamginsburg  20K Oct 23 05:50 ASDMBinary

that's it, there should be more files

d-l-walker commented 1 year ago

Yeah, I've encountered this exact issue (*.ms/SYSCAL does not exist) this week while helping a user with their data. Just had to re-download the data ...

keflavich commented 1 year ago 
$ ls -lhrt 2021.1.00172.L_uid___A002_Xfeb5e6_X4fa5.asdm.sdm.tar
-rw-r--r-- 1 adamginsburg adamginsburg 71G Oct 22 20:48 2021.1.00172.L_uid___A002_Xfeb5e6_X4fa5.asdm.sdm.tar
$ md5sum 2021.1.00172.L_uid___A002_Xfeb5e6_X4fa5.asdm.sdm.tar
49c35a99c57562657ac352d2c99e658e  2021.1.00172.L_uid___A002_Xfeb5e6_X4fa5.asdm.sdm.tar
keflavich commented 1 year ago

redownloaded

$ ls -lhrt *X4fa5.asdm.sdm.*
-rw-r--r-- 1 adamginsburg adamginsburg 71G Oct 22 20:48 corrupted_2021.1.00172.L_uid___A002_Xfeb5e6_X4fa5.asdm.sdm.tar
-rw-r--r-- 1 adamginsburg adamginsburg 70G Dec  2 06:05 2021.1.00172.L_uid___A002_Xfeb5e6_X4fa5.asdm.sdm.tar
keflavich commented 1 year ago

I re-extracted the more recent tarball today.

$ md5sum 2021.1.00172.L_uid___A002_Xfeb5e6_X4fa5.asdm.sdm.tar
ff74671933cacf5cb8ca45bbc97d1f3d  2021.1.00172.L_uid___A002_Xfeb5e6_X4fa5.asdm.sdm.tar
keflavich commented 1 year ago

Line cubes are good, continuum images are junk. Same issue noted in https://github.com/ACES-CMZ/reduction_ACES/issues/125#issuecomment-1430411878, most likely.

keflavich commented 1 year ago

Cubes & lines both look good; checked together on April 19 zoom call.

keflavich commented 1 year ago

TODO: experiment with robust=-2 because the beam is too large across the board.

d-l-walker commented 11 months ago

To-do: check whether the products are still size mitigated.

This issue is marked as done, but I don't see any PR, nor do I see the updated parameters in override_teclean_commands.json.

(Tagging #179 for tracking)

ksi85 commented 11 months ago

I tried robust=-2. This did not reduce the beam significantly. Limiting the uvrange to >25klambda did but the image looks meh:

image
pyhsiehATalma commented 9 months ago

截圖 2024-01-24 13 41 46

continuum of spw 33 line cube was not (fully?) subtracted in the pipeline version (using the existing cube under /working). For the combination of mosaic CS(2-1) I temporary fixed this issue with imcontsub. The imcontsub was applied with a sub-cube from -300 km/s to 300 km/s. No divergence was found.

betacygni commented 8 months ago

QA - Line contamination in continuum images from high/low frequencies

New image for continuum of spw33_35 is missing in Globus

d-l-walker commented 6 months ago

Reminder that SPWs 25,27,29,31,35 all need to be un-size-mitigated and re-stat-cont-ed @keflavich

keflavich commented 6 months ago

Moved files: mv *spw{25,27,29,31,35}* sizemitigated/. Rerun forthcoming.

keflavich commented 6 months ago

This one has caused a problem.

AssertionError: /orange/adamginsburg/ACES//data//2021.1.00172.L/science_goal.uid___A001_X1590_X30a8/group.uid___A001_X1590_X30a9/member.uid___A001_X15a0_X106/calibrated/working/uid___A002_Xfeb5e6_X4fa5_target_line.ms
$ ls -d /orange/adamginsburg/ACES//data//2021.1.00172.L/science_goal.uid___A001_X1590_X30a8/group.uid___A001_X1590_X30a9/member.uid___A001_X15a0_X106/calibrated/working/*ms/orange/adamginsburg/ACES//data//2021.1.00172.L/science_goal.uid___A001_X1590_X30a8/group.uid___A001_X1590_X30a9/member.uid___A001_X15a0_X106/calibrated/working/uid___A002_Xfeb5e6_X4fa5.ms
/orange/adamginsburg/ACES//data//2021.1.00172.L/science_goal.uid___A001_X1590_X30a8/group.uid___A001_X1590_X30a9/member.uid___A001_X15a0_X106/calibrated/working/uid___A002_Xfec857_X9f6.ms

It looks like the imaging parameters do not match the data.

betacygni commented 6 months ago

QA - Line contamination in continuum images from high/low frequencies

Summary: Could not find old image for spw25_27. New images look good, maybe a little bit too much structure in the spw25_27 which may resemble the structures that were seen in previous spw25_27 images when there was a little bit of line contamination, but it does not look wrong

Files checked: New: spw25_27 and spw33_35 Old: v1.1_20240314_high_spw33_35

Attached image: Top-left: spw33_35 Top-right: old_spw33_35 Bottom-left: spw25_27

Screenshot 2024-05-08 at 14 46 34