search

caracal-pipeline / caracal

Containerized Automated Radio Astronomy Calibration (CARACal) pipeline
GNU General Public License v2.0
29 stars 6 forks source link

astroquerry error in masking worker #560

Closed SpheMakh closed 5 years ago

SpheMakh commented 5 years ago

@Fil8 any idea how to fix this?

meerkathi - 2019-08-06 19:26:05,939 INFO - Running worker masking_worker
meerkathi - 2019-08-06 19:26:05,939 CRITICAL - INFO:STIMELA-9:Running job query_source_catalog
meerkathi - 2019-08-06 19:26:05,939 CRITICAL - INFO:STIMELA-9:STEP 1 :: Catalog pulled
meerkathi - 2019-08-06 19:26:05,988 CRITICAL - Traceback (most recent call last):
meerkathi - 2019-08-06 19:26:05,989 CRITICAL -   File "/home/sphe/work/github/meerkathi/workdir/venv/local/lib/python2.7/site-packages/stimela/recipe.py", line 815, in run
meerkathi - 2019-08-06 19:26:05,989 CRITICAL -     job.run_python_job()
meerkathi - 2019-08-06 19:26:05,989 CRITICAL -   File "/home/sphe/work/github/meerkathi/workdir/venv/local/lib/python2.7/site-packages/stimela/recipe.py", line 98, in run_python_job
meerkathi - 2019-08-06 19:26:05,990 CRITICAL -     function(**options)
meerkathi - 2019-08-06 19:26:05,990 CRITICAL -   File "/home/sphe/work/github/meerkathi/meerkathi/workers/masking_worker.py", line 107, in query_catalog_sumss
meerkathi - 2019-08-06 19:26:05,990 CRITICAL -     tab = p[0]
meerkathi - 2019-08-06 19:26:05,990 CRITICAL -   File "/home/sphe/work/github/meerkathi/workdir/venv/local/lib/python2.7/site-packages/astroquery/utils/commons.py", line 239, in __getitem__
meerkathi - 2019-08-06 19:26:05,991 CRITICAL -     return super(TableList, self).__getitem__(key)
meerkathi - 2019-08-06 19:26:05,991 CRITICAL - IndexError: list index out of range
meerkathi - 2019-08-06 19:26:05,991 ERROR - An unhandled exeption occured. If you think this is a bug please report it.
meerkathi - 2019-08-06 19:26:05,991 ERROR - Your logfile is here: /home/sphe/work/github/meerkathi/workdir/log-meerkathi.txt.
meerkathi - 2019-08-06 19:26:05,991 ERROR - You are running version: 0.1.0-1027-g6005012
meerkathi - 2019-08-06 19:26:05,991 ERROR - Traceback (most recent call last):
  File "/home/sphe/work/github/meerkathi/meerkathi/__init__.py", line 213, in __run
    pipeline.run_workers()
  File "/home/sphe/work/github/meerkathi/meerkathi/workers/worker_administrator.py", line 232, in run_workers
    recipe.run()
  File "/home/sphe/work/github/meerkathi/workdir/venv/local/lib/python2.7/site-packages/stimela/recipe.py", line 853, in run
    raise RuntimeError("An unhandled exception has occured. This is a bug, please report")
RuntimeError: An unhandled exception has occured. This is a bug, please report
KshitijT commented 5 years ago

I think this is sorted, @SpheMakh ? Can we close this?

SpheMakh commented 5 years ago

Fixed

HaoChen-ast commented 5 years ago

I meet the same issue. May I know do you sort it? I update MeerKATHI and Stimela on 12 Nov 2019 and use singularity and python3 on Ilifu. Thanks a lot!

KshitijT commented 5 years ago

@HaoChen-ast , if this is a Northern field (out of SUMSS area of coverage), then please change to: catalog : 'NVSS' in the masking worker, the current default is SUMSS.

HaoChen-ast commented 5 years ago

I am reducing MeerKAT data which should be a south field. The same IndexError when I use NVSS catalogue.

I tried SUMSS catalogue and another error:

meerkathi - 2019-11-13 15:14:17,610 CRITICAL - INFO:STIMELA-2:Recipe executed successfully meerkathi - 2019-11-13 15:14:17,623 CRITICAL - INFO:STIMELA-12:--------------------------------- meerkathi - 2019-11-13 15:14:17,623 CRITICAL - INFO:STIMELA-12:Stimela version 1.2.2 meerkathi - 2019-11-13 15:14:17,623 CRITICAL - INFO:STIMELA-12:Sphesihle Makhathini sphemakh@gmail.com meerkathi - 2019-11-13 15:14:17,623 CRITICAL - INFO:STIMELA-12:Running: Make masks meerkathi - 2019-11-13 15:14:17,623 CRITICAL - INFO:STIMELA-12:--------------------------------- meerkathi - 2019-11-13 15:14:17,706 CRITICAL - INFO:STIMELA-12:Adding Python job 'query_source_catalog' to recipe. meerkathi - 2019-11-13 15:14:17,706 CRITICAL - INFO:STIMELA-12:Adding Python job '1' to recipe. meerkathi - 2019-11-13 15:14:17,717 ERROR - An unhandled exeption occured. If you think this is a bug please report it. meerkathi - 2019-11-13 15:14:17,717 ERROR - Your logfile is here: /idia/projects/vela/1547413577/log-meerkathi.txt. meerkathi - 2019-11-13 15:14:17,717 ERROR - You are running version: 0.2.0 meerkathi - 2019-11-13 15:14:17,718 ERROR - Traceback (most recent call last):

KshitijT commented 5 years ago

From what I remember, galactic plane is not a part of SUMSS. ;)

HaoChen-ast commented 5 years ago

Thank for your reminding. I checked. NVSS does not cover the region I am working on, SUMSS do cover these regions. I will continuum to check how to sort the masking worker issue error.

HaoChen-ast commented 5 years ago

The exact error should be the last few lines:

meerkathi - 2019-11-13 16:09:29,017 CRITICAL - INFO:STIMELA-12:--------------------------------- meerkathi - 2019-11-13 16:09:29,017 CRITICAL - INFO:STIMELA-12:Stimela version 1.2.2 meerkathi - 2019-11-13 16:09:29,017 CRITICAL - INFO:STIMELA-12:Sphesihle Makhathini sphemakh@gmail.com meerkathi - 2019-11-13 16:09:29,017 CRITICAL - INFO:STIMELA-12:Running: Make masks meerkathi - 2019-11-13 16:09:29,017 CRITICAL - INFO:STIMELA-12:--------------------------------- meerkathi - 2019-11-13 16:09:29,219 CRITICAL - INFO:STIMELA-12:Adding Python job 'query_source_catalog' to recipe. meerkathi - 2019-11-13 16:09:29,220 CRITICAL - INFO:STIMELA-12:Adding Python job '1' to recipe. meerkathi - 2019-11-13 16:09:29,256 ERROR - An unhandled exeption occured. If you think this is a bug please report it. meerkathi - 2019-11-13 16:09:29,256 ERROR - Your logfile is here: /idia/projects/vela/1547413577/log-meerkathi.txt. meerkathi - 2019-11-13 16:09:29,256 ERROR - You are running version: 0.2.0 meerkathi - 2019-11-13 16:09:29,637 ERROR - Traceback (most recent call last): File "/idia/projects/vela/meerkathi-venv-python3-master-12Nov2019/lib/python3.7/site-packages/meerkathi/main.py", line 165, in __run pipeline.run_workers() File "/idia/projects/vela/meerkathi-venv-python3-master-12Nov2019/lib/python3.7/site-packages/meerkathi/workers/worker_administrator.py", line 244, in run_workers worker.worker(self, recipe, config) File "/idia/projects/vela/meerkathi-venv-python3-master-12Nov2019/lib/python3.7/site-packages/meerkathi/workers/masking_worker.py", line 487, in worker label='Mosaicing catalog') File "/idia/projects/vela/meerkathi-venv-python3-master-12Nov2019/lib/python3.7/site-packages/stimela/recipe.py", line 858, in add time_out=time_out) File "/idia/projects/vela/meerkathi-venv-python3-master-12Nov2019/lib/python3.7/site-packages/stimela/recipe.py", line 386, in singularity_job self.recipe.stimela_path, _cab.task), "/scratch/code", "ro") File "/idia/projects/vela/meerkathi-venv-python3-master-12Nov2019/lib/python3.7/site-packages/stimela/singularity.py", line 65, in add_volume "Path {0} cannot be mounted on container: File doesn't exist".format(host)) OSError: Path /idia/projects/vela/meerkathi-venv-python3-master-12Nov2019/lib/python3.7/site-packages/stimela/cargo/cab/montage.mosiac/src/ cannot be mounted on container: File doesn't exist

KshitijT commented 5 years ago

Essentially the above error occurs when the queries table is empty for some reason. If your source is covered by SUMSS, could you please try a query on Vizier for the SUMSS catalog and see if that gives any results? If it does then it is a different error than what @SpheMakh had initially.

KshitijT commented 5 years ago

@HaoChen-ast , this is a completely different issue, could you please open it separately?

KshitijT commented 5 years ago

P.S. I get the exact same error when run in default mode; but when I set up my config file options as:

- masking:
    enable: true
    query_catalog:
      enable: true
      catalog : 'NVSS'

(my source is in NVSS but not in SUMSS) and it works fine.

HaoChen-ast commented 5 years ago

If no sources in the catalogue of the reduced field, the IndexError issue of #560 will show. That's what I meet when using the NVSS catalogue.

Now, my issue is a new one now after address the catalogue to SUMSS. I have put a new issue #659

Thanks! @KshitijT