search

askap-vast / vast-pipeline

This repository holds the code of the Radio Transient detection pipeline for the VAST project.
https://vast-survey.org/vast-pipeline/
MIT License
7 stars 3 forks source link

Pipeline breaks when ra_uncertainty or dec_uncertainty is set to 0.0 #704

Open ddobie opened 4 months ago

ddobie commented 4 months ago

The latest post-processing code will account for the systematic uncertainty in the ASKAP positions, so we can now set ra_uncertainty and dec_uncertainty to zero.

However, this breaks the pipeline:

2024-02-06 04:33:05,969 runpipeline ERROR Processing error:
Catalog coordinates cannot contain NaN entries.
Traceback (most recent call last):
  File "/import/ada2/ddob1600/vast-pipeline/vast_pipeline/management/commands/runpipeline.py", line 340, in run_pipe
    pipeline.process_pipeline(p_run)
  File "/import/ada2/ddob1600/vast-pipeline/vast_pipeline/pipeline/main.py", line 225, in process_pipeline
    sources_df = association(
  File "/import/ada2/ddob1600/vast-pipeline/vast_pipeline/pipeline/association.py", line 1267, in association
    sources_df, skyc1_srcs = advanced_association(
  File "/import/ada2/ddob1600/vast-pipeline/vast_pipeline/pipeline/association.py", line 938, in advanced_association
    idx_skyc1, idx_skyc2, d2d, d3d = skyc2.search_around_sky(
  File "/import/ada2/ddob1600/.conda/envs/vast-pipeline-dev/lib/python3.8/site-packages/astropy/coordinates/sky_coordinate.py", line 1511, in search_around_sky
    return search_around_sky(searcharoundcoords, self, seplimit,
  File "/import/ada2/ddob1600/.conda/envs/vast-pipeline-dev/lib/python3.8/site-packages/astropy/coordinates/matching.py", line 365, in search_around_sky
    kdt1 = _get_cartesian_kdtree(ucoords1, storekdtree)
  File "/import/ada2/ddob1600/.conda/envs/vast-pipeline-dev/lib/python3.8/site-packages/astropy/coordinates/matching.py", line 472, in _get_cartesian_kdtree
    raise ValueError("Catalog coordinates cannot contain NaN entries.")
ValueError: Catalog coordinates cannot contain NaN entries.
CommandError: Processing error:
Catalog coordinates cannot contain NaN entries.

I have tested the same data with ra_uncertainty and dec_uncertainty set to 1.0 (the default) and 0.0, and the error only seems to occur with 0.0.

My assumption is that these values are converted to weights somewhere (i.e. 1/uncertainty) which then results in NaN values. Maybe here: https://github.com/askap-vast/vast-pipeline/blob/6209e0d7b1431068869cb917ed6c4e8501aa0c52/vast_pipeline/image/main.py#L435-L456

ddobie commented 4 months ago

Origin of this is that for whatever reason, some selavy sources have zero astrometric uncertainty. These should probably be padded in some way (e.g. at post-processing we should set a minimum astrometric uncertainty) but in the meantime it's possible to get around this by just setting the ra_uncertainty and dec_uncertainty to some small number.