search

askap-vast / vast-pipeline

This repository holds the code of the Radio Transient detection pipeline for the VAST project.
https://vast-survey.org/vast-pipeline/
MIT License
7 stars 3 forks source link

Runs using corrected data products fail #582

Closed marxide closed 2 years ago

marxide commented 2 years ago

The corrected data products differ in name from their original counterparts, so the pipeline sees them as different as intended. However, the measurement names (the Selavy component names) are the same and so when these corrected catalogues are ingested, the unique constraint on Measurement.name causes the run to fail if the uncorrected version of that catalogue has already been ingested.

Two solutions immediately come to mind:

  1. Remove the unique constraint on Measurement.name.
  2. Modify the component names in the corrected Selavy catalogues to be unique.

The former is far easier to do so I'm leaning towards that. I'm also not sure why the constraint exists in the first place, it doesn't appear to be required.

ajstewart commented 2 years ago

To give context I think the unique name was born out of knowing that every measurement should be unique combined with the lack of front facing ID values on the website. So the name took the place of the ID value so that users could enter the measurement name in the search field and get the measurement they want.

While I see the logic, I think the usage scenarios make this more annoying than anything. It doesn't really break anything if the name is the same and the rule is still enforced on the image. So I think removing the unique restriction on the measurement name is ok.

There is also something to be said for improving the actual naming convention that might be worth a separate issue: https://github.com/askap-vast/vast-pipeline/blob/995590b9008507a2591a8667a1c71eb9e93b4625/vast_pipeline/image/main.py#L347-L349

I don't think this is really smart enough for a unique restraint and also I have a feeling that the standard output from casda has changed since this bit of code was written.